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ABSTRACT 

Usiag an empirical investigation of alternate item 
nonresponse "adjustment procedures in a National Longitudinal Study 
(NLS) of missing a .3 faulty' data, it is indicated that in some cases 
imputation can reduce the accuracy* of survey estimates. A National 
Sample of the high school class of 1972 is designed to provide 
statistics on students moving into early adulthood. The bias 
resulting from nonresponse and response errors is evaluated using hot 
deck and weighting class adjustment techniques to analyze telephone 
followups after the original response to 20 critical survey items. 
Regarding bias variance, comparison is made with estimates when no 
imputation or editing techniques are used. Item nonresponse 
imputation and adjustment procedures ar£ presented with the design of 
the experimental data base. Bias, variance and mean square error 
estimations, and the results of the imputation comparisons are 
presented. The investigation of more complex statistical 'analyses is 
suggested. A bibliography is presented with appendices including an 
executive summary, data coding of the study, the followup 
questionnaire, statistical comparisons, and formulas for the standard 
errors program in the survey. (CM) 
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FOREWORD 



The National Longitudinal Study -(NLS) of the High School Class of 1972 is 
a larg t e-scale long-term survey effort supported by the National Center for 
Education in the Department of Health, Education, and Welfare (DHEW).> Broadly 
stated, NLS is designed to provide statistics on a national sample of students 
as they move out of the American high school system into the critical years of 
early adulthood. Data have been gathered ancf "colligated from several sources, 
coded and edited for analysis purposes, and stored on magnetic computer tapes 
for future access. The current t^pes contain base-year (1972) survey data, \ 
collected by the Educational Testing Service, integrated with first follow-up 
(1973), second follow-up (1974), and third follow-up (1976) survey data, 
collected by the Research Triangle- Institute (RTI). This tape package is 
augmented periodically as data. from subsequent follow-up surveys become avail- 
aMe. 

The merged NLS data file represents a rich and complex source of informa- 
tion. It contains important and timely data of potential use to a broad 
spectrum of educators, researcher^, policy analysts, and decision makers. 
Preparation of this data file for general use by such a large and diverse set 
of users was a difficult task. Decisions as to the kind and extent of data 
editing that needed to be performed were by no means clear-cut. In conflict 
were the need to maintain a faithful record of the original raw data — includ- 
ing respondent errors and inconsistencies — versus the need to provide a 
straight-forward set of data for the* typical researcher. The former need was 
\ given more importance in the approach. to editing the NLS data file. That is, 
it was felt that editing, recoding, imputation, or other data" transformation 
procedures should be minimized so as to limit, tacit assumptions about the 
respondents or subjective interpretation of the data that would be required 
for these procedures. Any imputations or modification of the original data 
might be considered "biased" by other investigators. Where imputations are 
concerned, researchers should be free to set data standards ia accordance with 
their specific needs. Thus, the data transformation procedures that were used 
were directed toward making the data available in a consistent and useful 
format that would preserve as accurately as possible the original responses of 
the study participants. s 

iii 



The file processing that was performed involved extensive verification, 
cleaning, and supplementary coding of the original data. Editing transforma- 
tions were limited to verifying that respondents followed the written instruc- 
tions and routing patterns in the questionnaire. If either type of instruc- 
tion was violated, then supplementary codes were* inserted in the data to 
indicate the location and nature of the violation. The extent and details of 
the editing are discussed in the ^ NLS Users Manual (Levinsohn, et al, 1978). 

It would have been possible to provide another level -of editing. Respon- 
ses could have been edited with respect to the content of the question and 
with respect to the interrelations of sets of questions. But this kind of 
editing requires some guidelines and is usually directed by the particular set 
of hypotheses under investigation. Since the data file is designed for such a 
broad range of investigators, and in the interest of timeliness, it was 
decided that any further editing must be left to the* discretion of the ana- 
lyst. This decision creates some additional work for the user, but it allows 
full access to all the original detail in the data file. No analyses have 
been prohibited by preprocessing or prior summarization of the data. 

The decision to produce the data file in this format places some addi- 
tional responsibility on the user. Since little simplification of the data 
has been done, the analyst must te,;ercise care in using the data file. The NLS 
data base is large and , complex. The routing patterns in the instruments and 
the fact that data have been collected .at different points in time require 
that the analyst view the data base holistically . Many of the individual 
items are not well suited to independent analysis. Moreover, it is necessary 
to consider the interdependent nature of such items and to study the patterns 
of response to those items. Given" consideration of these cautions and careful 
study cf the documentation provided in the Users Manual, the NLS dat^) file 
will serve as the rich source of information that it was designed to" provide. 

In this empirical investigation of alternate item nonresponse adjustment 
procedures, the nature of missing and faulty data was investigated based upon 
thef original responses to 20 critical items on the third follow-up instrument. 
Since these critical items were subject to telephone follow-up to correct 
missing and faulty data, the bias resulting from nonresponse and response 
errors could be evaluated. Further, two nonresponse imputation procedures 
were tested on this experimental data set of original responses to the 20 
critical items and "the resulting estimates of means and proportions compared 
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with respect to their bias, variance, and mean square error to the estimates 
obtained when no imputation or editing procedure is used. One of' the most 
striking findings of the study is that in some cases imputation can reduce 
rather than increase the accuracy of survey estimates. It is hoped that the 
results of this investigation will provide guidance to the user of the NLS 
data base in dealing with missing and faulty data. 

*An executive summary highlighting the major findings of this investiga- 
tion is presented in Appendix A. 
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I. INTRODUCTION 

* In 1968 the National Center for Education Statistics (NCES) conducted a 
survey to determine the specific data needs of educational policymakers and 
researchers* Respondents to the survey expressed a need for data that would 
allow comparisons of student educational and vocational experiences with later 
outcomes. This finding provided the impetus for NCES to begin planning for 
the first of an intended series of national longitudinal studies. 

In April 1970 a number of prominent educational researchers and admini- 
strators met with interested Federal officials in Washington, D. C. The 
National Longitudinal Study of the High School Class of 1972 (NLS) refleccs 
their guidance and the interests and data needs of a number of United States 
Office of Education (USOE) agencies: the Office of Planning, Budgeting, and 
# Evaluation; the Bureau of Postsecondary Education; the Bureau of Occupational 
and Adult Education; the Bureau of Programs for the Handicapped; as well as 
the National Center for -Education Statistics. Four advisory committees pro- 
vided guidance in the planning and implementation of the survey. One commit- 
tee was composed of research experts and representatives of various educa- 
tional organizations; two others were made up of officials of State education 
agencies; and the fourth, an internal USOE committee, represented' the data 
needs of the various offices and bureaus of the U. S. Department of Health, 
Education, and Welfare (DHEW). 

"The primary purpose of NLS is the observation of the educational and 
vocational activities, plans, aspirations, and attitudes of young people after 
they le^ve high school and the investigation of the relationships of this 
information to their prior educational experiences and biographical charac^ 
teristics. Ultimately, /the study will allow a better understanding of the 
development of students as they pass through the American educational system 
and of the complex factors associated with individual educational and career 
outcomes. Such information is essential as a basis for effective planning, 
implementation, and evaluation of Federal policies and programs designed to 
ennance educational opportunity and achievement and to upgrade occupational 
attainments and career outcomes. ' * ** 

Following 3 rather extensive period of planning, which included the 
design and field test of survey instrumentation and procedures, a full-scale 



survey was initiated in the spring of 1972- The sample design called for a 
deeply stratified national probability sample of 1,200 schools with 18 seniors 
per school, school size permitting. The* resulting base-year sample of 19,144 
students from 1,009 high schools provided base-year data on up to 3 data 
collection forms— a Test Battery (TB), a Student Record Information Form 
(SRIF), and a Student Questionnaire (SQ). The key form,vthe SQ, was completed 
by 16,683 seniors. . 

The first follow-up survey began in October 1973 and ended in April 1974. 
Added to the base-year sample were 4,450 1972 high school seniors from 257 
additional schools *that were unable to participate earlier. This brought the 
total first follow-up sample to 23,451 potential respondents. First follow-up 
forms were mailed to 22,654 scudents. There */ere 21,350 sample members , who 
completed a First Follow-Up Questionnaire, 69 percent by mail and 31 percent 
by personal interview. Of the 16,683 seniors who completed- a base-year Stu- 
dent Questionnaire, 15,635 took part in the first . follow-up 'survey — a sample 
retention rete of 93.7 percent. Participants were asked where they were in 

October 1973 and what they were doing with regard to work, education, and/or 
* 

training. Similar information was requested for the same time period in 1972 
to facilitate tracing of progress since leaving high school and to define the 
factors that affect that progress. 

The second follow-up survey began in October 1974, and was completed in 
April 1975, with forms sent to 22,364 potential respondents. There were 
20,872 sample members who completed a Second Follow-Up Questionnaire, 72 
percent by mail and 28 percent by personal interview. Cf the 21,350 persons 
who completed a First Follow-Up Questionnaire, 20,194 '94.6 percent) also 
responded to the second follow-up survey. 

The third follow-up survey began in October 1976 and ended in May 1977. 
Questionnaires were mailed to the last known addresses of the sample members 
whose addresses appeared sufficient and correct and who had not been removed 
from active status by prior refusal, reported death, or other reasons. Some 
20,092 sample members completed a Third Follow-Up Questionnaire: 80 percent 
by mail and 20 percent by personal interview. The overall response rate was 
approximately 92 percent of the initial 21,807 mailouts. The retention rate 
of second follow-up respondents was 93.9 percent, and the retention rate of 
those sample members who completed ail 3 previous student instruments was 94.7 
percfent. 



In the National Longitudinal Survey (NLS) follow-up studies, a planned 
sequence of reminder postcards, additional questionnaire mailings, reminder 
mailgrams, and personal interviews contributed to the Aigh instrument response 
rates that were obtained, NLS uses a weighting clafes procedure based upon 
classifier variables obtained from prior data collections to adjust sampling 
weights for instrument nonresponse. 

A more difficult type of nonresponse to deal with, especially in a mail 

survey such as NLS, is that of item nonresponse. A large number of the ques- 

\ 

tionnaires that are returned have one or more blank items. The level of. item 
nonresponse depends on the type of question and the information being solici- 
ted. For example, categorical questions typically have 3 smaller rate of* 
nonresponse than quantitative questions. Of the items examined in this inves- 
tigation, the two items requesting information on family income had the lar- 
gest rate of nonresponse with approximately 12 percent of the incoming ques- 
tionnaires having missing responses. Associated with the problem of item 
nonresponse is that of inconsistent or invalid responses and violations of 
.routing patterns (often referred to as skip patterns). This is an obvious 
source of bias which must also be considered before the data is analysed. For 
categorical questions, inconsistencies can represent a greater source of bias 
than nonresponse. 

The first step in dealing with all of the above cases of item nonresponse 
ana inconsistencies in the responses to different items may be to check the 
questionnaire to determine ■ if logical imputations can be made for the missing 
items or inconsistent items based upon the responses to other questions. If a 
logical imputation cannot be made, the next step might be to recontact those 
who failed to answer or incorrectly answered a question. NLS designated 
certain items as critical questions. If an individual did not give a valid 
response to a critical question, then he or she was telephoned and the missing 
item completed or the inconsistency resolved^ Obviously this is the best 
solution to the problem of invalid responses. However, because of the number 
of items on the survey instrument and the frequency of item nonresponse and 
inconsistencies , " this procedure could not be implemented for all items. An 
♦attempt to resolve inconsistencies and obtain missing information for every 
item on the instrument by recontacting the individual would have defeated the 
economy of the mail survey. Hence, the researcher utilizing the data needs to 
consider whether logical editing rules are needed to resolve or discard the 
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inconsistent^responses. Item aonresponse adjustment procedures may also be 
considered to adjust for .the bias that can arise due to differences between 
characteristics of individuals who respond or fail to respond to an item. 
Presently, NLS is making no adjustment for possible bias effects^ due to 
item nonresponse when computing survey estimates. That is, the sample means 
and proportions are the weighted respondent averages where the weights reflect 
the sample selection weight for the individual after adjustments are made 
within weighting classes for instrument nonresponse. This procedure, which 
doer not adjust or impute for missing responses to an item; but which instead 
.uses the estimated respondent mean, will be preferred to throughout this paper 
as the no imputation procedure. In the following sections of this report, a 
review of possible item nonrespouse imputation and adjustment procedures is 
presented and an empirical investigation of two of these that appear to be 
suited for NLS data is presented. This imputation study was conducted using 
NLS data in such a manner that the bias and variance of the resulting imputa- 
tion-based estimators could be evaluated and compared to the bias and variance 
of the estimates produced by the no imputation procedure. The problem of 
resolving inconsistencies was not a part of the investigation, but the effect 
on the bias of survey estimates resulting from using data with inconsistencies 
was determined. 



a 
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II • ITEM NONRESPONSE IMPUTATION AND ADJUSTMENT PROCEDURES 

When an individual fails to respond to an item, there is still much 
available information about him contained on the questionnaire. For NLS, 
classifier variables such as race, sex, high school curriculum, high school 
grades, and' parents 1 education are known about the item nonrespondent , as well 
as his responses to other items on this questionnaire t and prior question- 
naires. All of the following procedures divide the sample members into cate- 
gories based upon information that, is available for both groups: those who 
respond to an item and those who fail to respond. The assumption being made 
is that the responses of individuals within the same post-stratum cell "are 
relatively homogeneous and that those in different cells are more heteroge- 
neous . 

A. The Cold Deck Imputation Procedure 

The cold deck imputation procedure substitutes values from some previous 
census or survey for missing items on a questionnaire. Using this procedure, 
a cold deck is formulated by classifying, this previous data according to the 
categories that are used to classify the present data set (e.g., race, sex, 
income, etc.). For each category that is defined, a distribution of responses 
is constructed based upon the older set of data. In processing the new data, 
when a missing response is determined for an item on a questionnaire, the 
weighting class to which the individual belongs is ascertained and a response 
is selected at random from the cold deck distribution for that weighting 
class. This response is then imputed (substituted) for the missing response 
on the questionnaire. After all the questionnaires have been processed and a 
cold deck value imputed for each missing response on the questionnaires, the 
means and variances are computed in the usual manner, ignoring the fact that 
an imputation procedure has been used. * » 

An advantage of the cold deck imputation method is that missing responses 
can be imputed-\as the data are- being processed, ^ut the procedure also in- 
creases the variance to an extent not reflected In the estimated variance. 
Other criticisms of the cold deck imputation technique include the procedure's 
heavy reliance on the accuracy and currency of the older set of data with 
respect to the new set of data and the fact that information from the current 
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' data is not being used for imputation purposes (Chapman, 1976). This tech- 
nique is not feasible for most NLS item imputation since there is no older 
body of data on the same subject that could be used to create the cold deck 
distribution of responses for the weighting classes. 

i 

B. The Hot Deck Imputation Procedure 

The hot deck imputation procedure eliminates the criticism that the 
current data is not being used for imputation purposes. This technique is 
similar to the cold deck procedure in that it allows imputation of missing 
responses as the data is being processed. When timeliness is an important 
factor as in the Census Bureau's Current Population Survey, .this facet can 
become very important. 

To use the hot deck imputation procedure, the individuals completing the 
questionnaire must again be divided up into categories. An initial value is^ 
determined for each category in the hot deck based upon previous dati. As the 
new data is being processed, the category to which each individual belongs is 
detepnined. If the questionnaire being processed is complete, then that 
individual's responses replace the responses stored in the relevant category 
of the hot deck. Thus new responses are supplied for each cell of the hot 
deck as they appear in the data file. When a questionnaire is encountered 
with a missing item, the response in the same cell of the hot deck is imputed 
for the missing response. When all questionnaires have been processed and the 
missing data imputed, the means and variances are again computed in the usual 
manner without accounting for the effect of the imputation procedure (Chapman, 
1976). 

Since the hot deck technique is the most commonly used item nonresponse 
imputation procedure, it was decided that the quality of hot deck imputation 
should be assessed. One reason hot deck is used is its flexibility and the 
ease with which it can be implemented. However, Bailar and Bailar (1978) 
demonstrated that the hot deck procedure will cause an increase in the vari- 
ance of sample means compared to a procedure which ignores missing values in 
computing item mrans. The magnitude of this increase in variance and the bias 
reduction resulting from imputation was investigated in this methodological 
study! Another flaw in the hot deck technique is that variance estimates 
cannot be obtained analytically but must be estimated using some form of 
pseudo replication such as balanced* repeated replication (BRR) . Iu practice, 
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most users of the hot deck ignore the fact that imputation has occurred and* 



compute the variance in the usual^ manner. The resulting variance will typi- 
cally underestimate the true variance of the sample statistic. In this inves- 



results obtained when imputation was ignored. Other flaws in the hot deck 
procedure are that there is no probability mechanism attached to the assign- 
ment of missing values and that the same individuals responses may be used 
repeatedly to supply missing information, 

C. The Weighting Class Imputation Procedure 

. In the present analysis of NLS data, a weighting class adjustment pro- 
cedure is used to adjust for instrument nonrespoftse (i.e., when a question- 
naire's not obtained for a sample member). Basically, the weighting class 
adjustment procedure assigns sample members to weighting classes based upon 
information available for both respondents and nonrespondents . Within these 
weighting classes, an individual is assigned an adjusted sampling weight. 
Respondents within weighting class £ have their sampling weight (which is the 
inverse of their probability of selection) multiplied by the weight adjustment 
factor, WS(£)/WR(£), to produce the nonresponse adjusted sampling weight where 
WS(£) is the sum of the sampling weights for all sample individuals in weight- 
ing class £, and WR(£) is the sum of the sampling weights of the respondents 
in weighting class £. Nonrespondents are assigned adjusted sampling weights 
of zero. Sample estimates are then obtained using these nonresponse adjusted 
weights (Bailar, Bailey, and Corby, 1978), 

Such a weighting class adjustment procedure may also be used for item 
nonresponse and adapted to become an imputation technique. To digress for a 
foment, we should mention the advantage of an imputation procedure over a 
weight adjustment procedure to adjust for bias due to item nonresponse. The 
weight adjustment procedure for instrument nonresponse requires that an ad- 
justed weight Mft calculated for each respondent which is then stored on the 



data record for that individual. If a similar weighting class adjustment 
technique were to be used for item nonresponse, then a weight corresponding to 
every item on the survey instrument would have to be stored on each individ- 
ual's data record. Considering the hundreds of items on the survey instru- 
ment, this is clearly not desirable. However, an imputation procedure would 



tigation, BRR estimates of variances were ;computed and then compared with 
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substitute values for the missing responses ( according to certain criteria 
allowing the data to be processed in the usual manner without requiring weight 
| adjustments for nonresponse to each item. A ' 

For quantitative data, an imputation technique equivalent to the usual 
weighting class adjustment procedure would be to assign the average value of 
the responses in a weighting class to all nonrespondents in that weighting 
class. For qualitative data where categories of responses are reported, this 
technique Cannot be used. For example, a yes-no type^jquestion might be coded 
1 for yes and 0 for no, but to impute 'the average (which might be something 
such as 0.2) to nonrespondents i§ not reasonable and also does not allow the 
usual tabulation of the data. To impute categorical responses,- a technique 
was developed for this investigation that is analogous to weighting class 
adjustments. For each item, this "weighting class 11 imputation procedure first 
determines the weighted response option proportions from all responding mem- 
bers of the various weighting classes. For. instance, the proportion of all 
respondents ia weighting class £ who make response k could be denoted p(k £). t 
Next, the sum of the sample weights for the nonrespondents who belong to 
weighting class £ is found; denote this sum by WN(£). The response k is then 
randomly imputed to nonrespondents in weighting class £ so that their sample 
weights sum to 

WN(£k) = p(k £) WN(£) . 

The nonrespondents in class £ to which the response k* is to be imputed is 
determined ia the following manner. First, list the nonrespondents with their 
sample weights in random order. Go down the list, summing weights until the 
sum equals WN(£k) . Impute the response k to the corresponding sample members. 
Continue this procedure until all nonrespondents in weighting class £ have had 
a response imputed for the, missing item. The estimated proportions making 
each response would then be determined in the usual way. Estimates of re- 
sponse level proportions will be the same as that which would have resulted 
had a weighting class adjustment procedure been used, except for weight accu- 
mulations that cannot be expected to break precisely at the place desired. To 
the extent that this categorical imputation technique produces the same esti- 
mated response proportions as a weighting class adjustment procedure, variance 
approximations for the weighting class adjustment procedure should be appro- 
priate for the imputation-based procedure. This is different from the hot 
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deck procedure for which an' analytical expression | for the variance is not 
available. 



D. 



The' Creation of Weighting Classes for Use in Imputation 



Before, implementing a weighting class imputation procedure, the sample 



must be partitioned into classes. The hot deck and cold deck imputation 
procedures can also perform more efficiently if the sample is divided into 
classes before -imputation occurs. In both instances, characteristics must be 
identified which define weighting^jclasses which vary with respect to response 
rates and survey estimates. For this in astigation, the weighting classes 
were based upon the student's race,, sex, high school grades, high school 
curriculum, and parents' education. Although these weighting classes were 
initially created to adjust v for total questionnaire nonresponse, it was felt 
that they would also be applicable in this investigation because of the large 
number and diversity of iteips studied. The analyst uJing only a few items 
from the'NLS data file should construct special weighting classes before im- 
puting for missing values. The remainder of this section discusses procedures 
that may be used to form weighting classes. 

In constructing weighting classes, the overall goal is to form* classes 
for which the responses within classes »are homogeneous and heterogeneous 
between classes and for which the response rate varies. Further, the charac- 
teristics used to define the weighting classes must be known for both re- 
spondents and nonrespondents. The choice of survey characteristics* to use 



should reflect the following ideas: 

1. Usually more gain results from the use of the coarser division 
y of several variables tnan from finer divisions of one. 

2. There is no need for completeness or symmetry in forming cells. 
Smaller cells may be combined. 

3. Different criteria may be used for different subgroups. It ma$ 
be decided to partition males with respect to different charac- 
teristics than females. 

4. The classifying variables should be unrelated to each other. 
If two variables are highly correlated, 'then either will de- 
scribe approximately the same amount of variation. 




To summarize then, the weighting- classes that are formed should be as 
different as possible; this also corresponds to having classes inten^ally as 
^ homogeneous as possible. Cluster analysis is particularly appropriate both 
for choosing the best set of classifying variables and for determining how 
each variable chosen should be used to subdivide the units into classes. A 
clustering technique that mky prove effective in this regard is Automatic 
Interaction Detection (AID) Analysis (Hartigan, 1975). The AID algorithm 
operates "by successively dichotomizing the sample according to the factor 
level (classifying variable) that minimizes ( the within weighting class sum of 
squares for the dependent variable. The result is a "tree" of clusters 
(weighting classes) having similar dependent variable values where the clus- 
ters are defined by tjie levels of the* factors selected in the computing algo- 
rithm. 
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III. THE DESIGN OF THE EMPIRICAL INVESTIGATION 

In considering the implementation of a nonresponse imputation procedure, 
one needs to be concerned about the quality of the resultant statistics, 
Since nonresponse imputation procedures are proposed to reduce the bias that 
may result from the different characteristics of respondents and nonrespon- 
dents, the amount of bias lir-the estimates when one of these procedures is 
used is crucial. This cannot be determined in any useful analytic fashion 
because the bias will depend on whether the assumptions underlying the adjust- 
ment technique hold. For instance, all of the techniques discussed in the 
previous section assume that the respondent and nonrespondents within each 
weighting ! class have responses that are similarly distributed. The extent to 
which respondents and nonrespondents differ will influence the degree to which 
the resulting estimators are biased. Other factors of importance ^re the 
degree to which responses within weighting classes are homogeneous and the 
extent to which response rates differ between weighting classes. 

At the present time no item nonresponse imputation procedure is being 
used for NLS data. This empirical investigation studied two nonresponse 
imputation techniques that appeared to be suited for NLS data. In this inves- 
tigation, a weighting class imputation procedure and a hot deck imputation" 
procedure were compared with respect to bi^.s, variance, and mean square error 
to the no item nonresponse imputation approach. \he bias and mean square 
error were determined based upon the results using a data set in which missing 
responses were secured by follow-up efforts. Th< following sections discuss 
the construction of the experimental data set^ for this investigation and the 
utilization of this data set. 

A. Construction of the Experimental Data Set 

To assess the bias associated with each of these nonresponse adjustment 
procedures, a data set wag needed in which item nonresponse occurred for some 
of the sample members and where the missing item responses could then be an- 
alysed using first the weighting class imputation technique, second the hot 
deck imputation technique, and finally the no imputation technique.: These 
results could then be compared to that obtained when the missing responses are 
added to the data set via telephone follow-up. 
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Rather than constructing a data set with artificially induced nonre- 
^ spoase, the decision was made to use actual data that contained item nonre- 
sponse for which the answers were subsequently obtained by telephone follow-up 
activities. By using data with naturally occurring patterns of item aonre- 
sponse, it was felt that a better understanding could *be obtained of the 
actual problems associated with item nonresponse and the effect of nonresponse 
adjustments on the precision of the resulting estimators. Such a data set was 
constructed from the NLS Third Follow-up (TFU) Survey by taking account of che 
following set of special circumstances. Certain items on the questionnaire 
were designated critical items by NLS staff. When an incoming questionnaire 
had a missing response or an inconsistent set of responses for one or more of 
these critical iter***, the questionnaire was marked as having failed edit. 
Subsequently, the individual involved was telephoned and the missing re- 
sponse(s) were added by the telephone operator. The data records for individ- 
A uals whose questionnaires failed edit contained the responses to these criti- 
cal items, but the records did not indicate what the original responses were 
or which responses were obtained by telephone editing. 

In order to obtain this information on the responses before telephone 
resolution, the questionnaires were re-examined by data editors and the orig- 
inal responses to the selected critical items were recorded. In all, a total 
of 10,850 questionnaires failed edit. For reasons of economy, a subsample of 
size 5,854 was selected for re-examination. The following 20 items were 
chosen to be representative of the types of items on the NLS instrument. 
These were examined on each of thj selected questionnaires: TQ1, TQ9, TQ10, 
TQ12, TQ15, TQ16, TQ29, TQ33, TQ517I TQ52, TQ66, TQ89, T(J90,- TQ101, TQ102, 
TQI18, TQ129, TQ131, TQ136\and TQ141. \ 

Except for TQ15, TQ16,N&89, and TQ141, the selected items have cate- 
gorical responses with TQ1, TQ9, TQ131, and TQ136, allowing the student to 
choose multiple response options. Item TQ10 was the lead-in question to a 
routing pattern with TQ12 to be answered by the unemployed and TQ15, TQ16, and 
TQ29 to be answered by the employed. Another major roifting pattern was con- 
trolled by TQ51, which directed those not attending school in/ 1974-1976 to 
skip to item TQ98. Other items found within routing patterns were TQ33, which 
had as its lead-in item TQ32, TQ102 vith lead-in item TQ101, and TQ131 and 
TQ136 with lead-in item TQ129. Finally, the four continuous items, TQi5, 
TQ16, TQ89, and TQ141, requested hours worked, weekly salary, college ex- 

-12- 



penses, and annual income. These questions were more sensitive and, histori- 
cally, these types of questions exhibit higher rates of item nouresponse. The 
text of all of. the selected items may be found in Appendix C. 

The subsample of TFU questionnaires that failed edit was examined. A 
working file was constructed containing the student identification code and 
additional codes to indicate whether or not each of the 20 items failed edit 
and, if so, what the original answer was. The coding used by the data editors 
is presented in Appendix B. The working file was then merged with the NLS 
Third Follow-Up File to create the two data files that were used in this 
investigation. The first file (hereafter referred to as the data file of 
telephone corrected and completed information) which was abstracted from the 
NLS Third Follow-Up File, contains the data records for those students who 
passed edit, combined with the subsample of those who failed edit. This file 
contains responses to the items after telephoning was used to replace missing 
responses and correct inconsistent responses. The second file, referred to as 
the pre-telephoning file, contains th/e responses to the 20 questions before 
telephoning was used to correct the<Cdata set. Since only a sample of the 
fail-edit mail questionnaires were -included in- the investigation, the sampling 
weights on the data records corresponding to these individuals were adjusted 
so that the sum <yt tjie weights of the subsampled mail questionnaires\that 
failed edit for each weighting class equaled the sum of the weights of all the 
mail questionnaires that failed edit from that weighting class. No weight 
adjustments were needed for the data records corresponding to those question- 
naires which passed edit or those completed by personal interview since all of 
these questionnaires were included in the study. 

v 

B. Response Error Rates in the Experimental Data Set. 

For each of the selected 20 items, the status \of the response before 
telephone follow-up was determined. A summary of the status of the original 
responses to these items for the sample of fail edit questionnaires is given 
in Attachment B-2 of Appendix B. These results were based upon the subsample 
of size 5,854 drawn from the 10,850 questionnaires that failed edit. Adding 
in the 9,235 questionnaires which passed edit (and hence had complete, con- 
sistent responses to the critical items) and adjusting to account for the 
sampling of fail-edit questionnaires yields the estimated response error rates 
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for the full NLS sample presented in Table 3-1. Since the experimental data 
set had the sampling weights of fail-edit questionnaires adjusted to account 
for the subsampling, these error rates also apply to the experimental data 
set. Except for 2 multiple response option questions (TQ1 and TQ9) and 4 
financial questions (TQ89HA, TQ89HB, TQ141FA, and TQ141FB), 95 percent of the 
questionnaires contained a response for an item that was consistent with other 
responses on the questionnaire. The highest rates of missing or blank re- 
sponses were found for the income items, TQ141FA and TQ141FB, with about 13 
percent nonresponse. Items TQ1 and TQ9 had the highest inconsistency rates; 
that is, the responses to TQ1 and TQ9 were most frequently in conflict with 
other questionnaire items. The "other 11 category in Table 3-1 is composed of 
those who failed an item but could aot be contacted for telephone resolution. 

From examining Table 3-1, one can verify that the data set that could be 
constructed of the original responses for vh* full NLS sample to the 20 crit- 
ical items would have a relatively small rate of item nonresponse and a some- 
what larger rate of inconsistent responses. Since the experimental data set 
contained tfnly a half of the fail-edit questionnaires with weight adjustments 
to account for fche subsampling, these rates also apply (with respect to their 
effect on survey estimates) to the experimental data set as well. However, in 
the physical process of imputing d£ta, the actual number of aonrespondents to 
each of the 20 items becomes important. This information is presented in 
Table 3-S>. When one considers that the experimental data set contains 15,089 
records, the number of nonrespondents is quite obviously of little practical 
importance except for the income items, TQ141FA and TQ141FB. 

C. The Utilization of *he Experimental Data Set 

The data set of original responses whose construction was discussed in 
previous sections was used to evaluate the hot deck and weighting class im- 
putation procedures with respect to the no imputation procedure. In o^rder to 
understand the results of the investigation, ietails are needed as to how the 
techniques were implemented. This section will discuss the logical editing 
used before imputation of missing values, the hot deck and weighting class 
procedures as implemented in this investigation, and the estimation of the 
bias, variance, and mean square error of the estimates. 
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Table 3-1. — Classification of original responses to the twenty 
selected- critical items 



Original response 

Item 

Consist? at Blank Inconsistent Other 



Discrete 










TQ1 


92.4 


0.2 


6.7 


0.7 


TQ9 


87.1 


0.3 


11.7 


1.0 


TQ10 


^96.0 


0.9 


2.'7 ■ 


0.4 


TQ12 


97.3 


0.5 


2.0 


0.3 


TQ29 


99.0 


0.5 


0.3 


' 0.2 


TQ33 


95.1 


0.5 


4.0 


0.4 


TQ51 


97.5 


0.9 


1.2 


0.4 


TQ52 


96.1 


0.9 


2.6 


0.4 


TQ(j6 


94.5 


1.1 


4.0 ■ 


0.5 


TQ90 


97.3 


.1.4 


0.9 


0.4 


TQ101 


98.5 


1.0 


0.2 


0.3 


TQ102 


/9T.1 


0.2 


0.5 


0.2 


TQ118 


197.1 


2.4 


0.1 


0.4 


TQ129 


97.2 


0.5 


0.1 


0.1 


TQ131 


99.5 


0.2 


0.2 


0.1 


TQ136 


99.6 


0.2 


0.1 


0.1 


Continuous 










TQ15 


98.6 


0.6 


0.5 


0.3 


TQ16 


97.3 


1.8 


0.6 


I 0.4 


TQ89HA 


92.9 * 


2.6 


3.8 


0.7 


TQ89HB 


92.5 


2.8 


4.0 


0.8 


TQ141FA 


82.9 


12.9' 


2.4 


V" 


TQ141FB 


82.5 


12.9 


2.7 


1.9 



Table 3-2. — The number of nissing responses for 
each iten in the experimental data 
set 



Item , Number of missing responses 



TQ1 


1 £ 




51 


TQ1U 




tJY\ 1 0 


CI 

j1 


TQ13 


AA 


TQlo 


109 


•pnofl 


J7 


TQJJ 


5/. 
JH 


*rr\C i 


yo 


TQ3Z 


0 ^ 


TQoo 


11/. 


TQoyHA 


9QA 
ZOO 


TnOQun 
TtJoyHB 




TQ90 


iso 


TQ101 


104 


TQ102 


25 


TQ118 


258 


TQ129 


57 


TQ131 


23 


TQ136 


22 


TQ1A1FA 


974 


TQ1A1FB 


1030 



1. Logical Editing Before Imputation of Missing Values 

At the start of this* investigation, a decision had to be made con- 
cerning the procedures to deal with inconsistencies. As the previous section 
demonstrated, inconsistent data occurred more often than missing data and for 
some items was a much more serious source of error. Judging from where the 
inconsistencies occurred, the major problem other than with TQ1 and TQ9 ap- 
peared to be associated with the routing pattern questions. Thus in recon- 
structing the data set, the decision was made to leave inconsistent data as 
observed rather than to code inconsistent items as blank. 

In computing the no imputation estimates, no attempt was made to 
force consistency on the data within records. However, the hot deck imputa- 
tion program and the weighting class imputation program included provisions to 
force consistency on data associated with routing patterns by requiring that 
the responses Within a routing pattern agree with the lead-in question to the 
routing pattern. When the responses within a routing pattern disagreed with 
the lead-in question, all of the responses within the routing pattern were 
coded as missing except for items which should have been skipped. These were 
coded as legitimate skips. For example, a common error was for an individual 
to respond "4" to TQ10 (indicating that he was not employed the first week of 
October 1976) then to respond to TQ12 (indicating whether or not he was look- 
ing for work), and then, instead of skipping TQ15, TQ16, and TQ29 as instruc- 
ted, to respond to these items, often giving nonsensical answers to these 
items which requested hours worked per week, weekly salary, and whether the 
individual was seeking a second job. In this case, TQ15, TQ16, and TQ29 would 
be recoded to legitimate skips and the response to TQ12 would ^e recoded to 
missing (since this response is also in doubt), 

2, Implementation of the Hot Deck Imputation Procedure 

The hot deck procedure was relatively easy to program and inexpen- 
sive to run with respect to computer time. Before using the hot deck imputa- 
tion procedure, the data file was sorted into 87 weighting classes and then 
according to sample design strata and school within strata. The weighting 
classes which were based upon the student's race, sex, high school grades, 
high school curriculum, and parents' education were originally formed for 
total .questionnaire nonresponse adjustments. These -weighting clases were 
adapted for item nonresponse imputation by incorporating certain routing 



pattern lead-in questions. In general, an initial hot deck was formed for 
each weighting class by going through the data file and recording the first 
completed response to each item. As the new daca was processed, the weighting 
class to which -each individual belonged was determined. If the item being 
examined was complete, that individual's response replaced the response stored 
in the hot deck for that weighting class. Thus, new responses were supplied 
for the hot deck as they appeared in the data file. When a questionnaire was 
encountered with a missing item, the response in the hot deck for that weight- 
ing class was imputed for the missing response. 

To adapt this procedure for items contained within routing patterns, 
the following rules were applied. For all items within a routing pattern, the 
weighting class or cell to, which an individual belonged was based upon the NLS 
weighting class and on his response to the lead-in item. If he responded in a 
consistent fashion to the lead-in item and the items within the routing pat- 
tern, then his responses to these items were entered as a group into the hot 
deck. Thus, in imputing for items within a routing pattern, either all of the 
items within the routing pattern were consistent and complete, in which case 
they entered the hot deck, or one or more of the items were missing or incon- 
sistent, in which case the responses to the group of items within the routing 
pattern were replaced by responses stored in the hot deck for the same NLS 
weighting class and the same response to the lead-in question. When the 
response to the lead-in item was missing, responses for the lead-in item and 
all items within the routing pattern were imputed as a block from the hot deck 
for the same weighting class as that to which the individual belonged. • 
3. Implementation of the Weighting Class Imputation Procedure 

The second item nonresponse imputation technique investigated was a 
procedure which we refer to as "weighting class" imputation. For quantitative 
variables, the weighting class imputation procedure simply replaced missing 
values by the estimated respondent mean for the weighting class containing the 
individual. When a mean or total is being estimated, this weighting class 
imputation technique results in the same estimate as that obtained when weight 
adjustments are made within weighting clases. For categorical items with 
missing responses, the weighting class imputation technique randomly assigned 
responses in such a manner that within each weighting class the weighted pro- 
portion of nonrespondents assigned each response option was equal (as far as 
possible) to the proportion of respondents who gave that response. Esti- 
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mates of response option proportions will be only approximately the same as 
those which would have resulted ha.d a weighting class adjustment procedure 
been used, since the weight accumulations will not break at precisely the 
point desired. To adapt this procedure for items within routing patterns, the 
weighting class an individual belonged to was determined by the NIS weighting 
class to which he was assigned and by his response to the lead-in question to 
the routing pattern. If the lead-in item was not answered as well, a response 
was first imputed for the lead-in question and then^responses were imputed for 
the items within the routing pattern. 

As an example of the use of the weighting class imputation pro- 
cedure, suppose that an individual from weighting class 14 failed to respond 
to TQ10, the lead-in item, and to TQ12, TQ15, TQ16, and TQ29, items within the 
routing pattern controlled by TQ10. The text of these questions is given in 
Appendix C. Essentially, TQ10 asked an individual's employment status with 
TQ12 to be answered by those unemployed and TQ15, TQ16, and TQ29 to be an- 
swered by those employed. In order to insure consistency of the responses to 
routing pattern questions, the lead-in question was imputed first if the 
response was missing. Since TQ10 is a categorical item, the randomization 
procedure described previously would be used to determine which one of the 
four responses to impute. For TQ10, the weighting classes used were the 
original NIS weighting classes used to adjust for instrument nonresponse. In 
this hypothetical case the individual belongs to NLS weighting class 14, so 
the imputation procedure would proceed as discussed previously for all indi- 
viduals from this weighting class with missing responses. Suppose that the 
response "2" was imputed for TQ10 for this individual indicating^ that he was 
employed part-time; as a result, TQ12 would be automatically coded as a legit- 
imate skip since this item was to be answered only by those unemployed. Since 
TQ15 and TQ16, which request hours worked per week and weekly salary, have 
continuous responses, the estimated respondent means for TQ15 and TQ16 (com- 
puted for the set of individuals from weighting class 14 who responded "2" to 
TQ10) would be imputed for the missing responses to TQ15 and TQ16. Now since 
TQ29 is within the routing pattern controlled by TQ10, the proportions re- 
sponding "1" and ,, 2" to TQ29 would be estimated, based upon the respondents 
from weighting class 14 who responded "2" to TQ10. Then response options "1" 
and ,, 2" would be randomly assigned to all the nonrespondents to TQ29 from 
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weighting class 14 who 'responded "2 n to TQ10 in such a manner that the weight- 
ed proportion of nonrespondents assigned each of the 2 responses will equal, 
as far as possible, the correponding weighted proportions from TQ29 respon- 
dents in weighting class 14 who responded "2" to TQ10. 

D. Estimation of the Bias Associated With Survey Estimates 

By using actual data that contained item nonresponse for which the an- 
swers were subsequently obtained by telephone follow-up, the bias could easily 
be estimated for the two imputation procedures .and the no imputation method as 
the difference between the estimates obtained using the imputation (or no 
imputation) procedure on the pre-telephone data set and the estimates obtained 
using the data set which had been corrected by telephone interviewing. Since 
the pre-telephone data set contained inconsistent responses as well as missing 
responses, the bias that was obtained reflected these two response error 
sources and, hence, contained nonresponse bias and measurement error bias due 
to inconsistent data. As will be discussed later, it was possible to parti- 
tion out this response error bias caused by inconsistent answers. The magni- 
tude of this response bias establishes an upper limit on the effectiveness of 
the imputation procedures since they were designed primarily to reduce the 
nonresponse or missing data component of the overall bias. 

E. Estimation of the Variance Associated With Survey Estimates 

The variance of the sample means and proportions was estimated using a 
variation of the balanced repeated replication (BRR) technique proposed by 
McCarthy (1966) for estimating the variance of complex survey statistics. BRR 
utilizes a balanced set of half samples to compute the sampling variance of 
these statistics where the variability of the replicated estimates approxi- 
mates the variance of the full sample statistic. In this investigation, the 
item nonresponse imputation procedures were separately applied to the balanced 
half samples to obtain BRR variance estimates which reflect the variability 
induced by the imputation procedures. / \ 

As mentioned previously, most users of imputation procedures ignore the 
fact that imputation has occurred when computing the variance of survey es- 
timates. The reason for this is that computing accurate variance approxi- 
mations which allow for the fact that imputation occurred is both difficult 
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and costly. While we feel that pseudoreplication methods should provide valid 
estimates of the variance induced by imputation, the standard software pack- 
ages for these procedures do not allow one to employ the imputation procedure 
on each individual half sample before computing >the half sample estimates. As 
discussed in Appendix D, the imputation procedure must be applied indepen- 
dently to the half samples -in order to estimate the variance induced by the 
imputation procedure. Thus some software development would be necessary to 
create one's own or to modify existing packages before the variance of impu- 
tation-based statistics could be estimated. Among the objectives of this 
investigation was the determination of the effect of imputation on the vari- 
ance of survey estimates and the underestimation effect caused by ignoring 
imputation in computing variance estimates. A discussion of. the theory 
underlying the balanced repeated replication procedure is given in Appendix D 
as well as a justification for using the procedure in the manner specified to 
compute the variance of imputation-based statistics. 

To estimate the variance when imputation was ignored, STDERR, an RTI 
software package, was used. This package utilizes a Taylor Series lineari- 
zation to estimate the variance of complex survey estimates (see Appendix E 
for the specific variance estimation formulas). For the ratio estimates of 
means and ' proportions used exclusively in this investigation, the Taylor 
Series expansion of the variance results in the usual estimate for the var- 
iance of a ratio estimate as given in standard sampling texts (e.g., Cochran, 
1977, pp. 153-154). As a side issue of this investigation, the STDERR and BRR 
variance estimates were computed for the surve/ estimates obtained when no 
imputation was used. In this situation, the two procedures were measuring the 
same quantity so that the two variance estimates could be compared. 

F. Estimation of the Mean Square Error Associated with Survey Estimat es 

As discussed in Section III~D, the bias induced by item nonresponse and 
inconsistencies was defined to be the expected value of the difference between 

A 

the imputation-based (or no imputation) estimate (p) and the corresponding 
estimate (p^) obtained when the telephone corrected and consistent follow-up 
data are used. The magnitude of this bias was estimated by 

A A A 

BIAS (|J) = |J-|J T 
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The mean square error (MSE) of the statistic (p) was defined to be the ex- 
pected value of the squared difference of the estimate from the value obtained 
using the post- telephone data which was estimated as 

MSE (m) = (Bias(M)] + Var^M) 
Note that this estimate witj. be biased to the extent that the correlation 

A A 

between \i and is different from unity. The reason for this is that 

E[Bias(u)] 2 = [Bias(M)] 2 + Var(M) + Var(M t ) - 2 Cov(m,M t ) 

A A 

Since [J and \}^ are estitoated from largely the same data set (except for mod- 
erate nonresponse) one wduld expect their correlation to be close to unity. 
With this anticipated high correlation, the joint contribution of the extra 
terms 

A • A A A 

Var(M) + Var(M T ) - 2 Cov(m,u t ) 
should be negligible. 
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IV. RESULTS OF THE EMPIRICAL INVESTIGATION 



The question of tKe magnitude of response errors and their ,effect on the 
quality of survey estimates is of great importance to users of the NLS survey 
data. Except for about 60 critical items on the survey instruments (approxi- 
mately 1/3 of the total items), the data are not edited to replace missing 
data or resolve* inconsistencies. As with the use of any survey instrument 
which requests the same or similar information in more than one item, incon- 
sistencies between the responses to similar items occur and indicate the 
presence of measurement error. The effect of this source of response error on 
the quality ot survey estimates is of concern, as well as the effect of miss- 
ing responses. Some questions that should be of concern f ,o the researcher are 
the following: / 



1. What is* the magnitude of the bias resulting from these sources 
of response error? I * • i 

2. Is editing or deletion needed to reduce, measurement error 
caused by inconsistent responses? 

* 3. How serious is the error caused by nonresponse? 

4. If a nonresponse bias imputation procedure is needed, which one 
would be "best" for the type of itemi being examined? 

In this study, these questions were investigated for 20 critical items that 
were selected a;; representative of the various types of items found on the NLS 
instrument. After conversations with'RTI's data editing staff, those key 
items that the data editors felt had the most missing or inconsistent data 
were selected. While the items studied in this investigation are similar in 
format to other items on the instrument, the frequency of response errors may 
be greater among the study items since the anticipated worst cases were se- 
lected. The original responses before telephone editing were used to form the 
data base for the investigation, and the responses after telephone resolution 
were used to judge the quality of the data set and the performance of the 
imputation procedures. The effect of response errors in the original data and 
the efficacy of the two common nonresponse adjustment procedures — hot deck and 
weighting class--are discussed in the remainder of this section. 
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A. R esponse Error Biea in the Experimental Data Set 

Estimates of population values from sample surveys are subject to two 
kinds of error. Variable error is the random portion which includes sampling 
error attributable to the random selection of individuals rather than a com- 
plete census of all individuals, and also variable measurement errors due to 
the natural fluctuations in questionnaire responses and data transcriptions. 
The second type of error is bias which is a systematic error that can result 
from the estimation procedure, nonresponse, or nonsampling errors inherent in 
the measurement process. One may model the total error associated with using 
the sample estimate 6 to predict the population parameter 8 by 

i 

e - e = [e - E(e)] + [E(e) - e] 

where E(0) is the expected value of 'the statistic 6 over a conceptual sequence 
of repeated samplings and repeated interviewing-transcribing trials for a 
given sample. The first terra represents the variable errors; the second teqn 
represents the bias. The mean ^uare error (MSE) is defined as the expecta- 
tion of the squared total error and is given by 

E[9 - 6 ] 2 = E[8,- E(0> ] 2 + [E(6) - 0] 2 ^ 

2 

MSE = Var(6) + [Bias] 

The first w .rm of the MSE is the sampling variance of the estimate 6 ; the 
second term is the square of the bias of the estimate. 

The ultimate problem of the sample designer is to construct a sample 
survey in such a way that the total error of the results is minimized (not 
just the random sampling error). This means that concern must be given to the 
protocol for collecting the data, the design of the survey instrument, and the 
number of individuals to be surveyed. Similarly, in utilizing the survey 
data, the investigator must also be concerned with the total error. After 1 the 
data is collected, however, the variable or sampling portion of the error is 
fixed. In this case, the investigator should.be concerned with the extent of 
instrument and item nonresponse, and with the amount of systematic measurement 
or response error that is evident in the data. Various imputation procedures 
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are available to reduce the bias caused by nonresponse, and logical editing 
and/or deletion may be used to resolve inconsistencies which are symptomatic 
of response or measurement errors in the data. 

The effect of bias when drawing inferences from survey data can be illus- 
trated by examining the effect on interval estimates and their associated' 
confidence coefficients. When bias is present, the entire sampling distribu- 
tioc of 9 about 8 is displayed by an amount equal to the bias. This results 
in a distortion of the areas in the tails of the sampling distribution of the 
appropriate test statistic. For example, suppose (9 - Bias - 8 )/ s q 
is distributed as Student's t, in which case the proper two- tailed confidence 
limits for 6 with confidence coefficient a are given by 

i + ts:iti-^] 

tb e 

where t is the value of the t statistic at 1 - a/2. If the bias lis positive, 
the standard interval represented by the confidence limits 

e ± t S q 

will exceed 8 more frequently than it should. Negative bias will have the 

opposite effect. For each increase of 0.5 in the absolute bias ratio, 

( Bias /Si), the probability that the corresponding interval fails to contain 
8 

8 will be approximately doubled. If, for example, the bias ratio for a sample 
statistic is 1.0, a presumed 95 percent confidence interval would have a 
chance of about 20 percent of excluding 9 instead of a 5 percent chance. 

In most surveys, the bias of sample estimates is not fcnown. However, in 
this empirical -.investigation, the bias in survey estimates due to using data 
containing item nonresponse and items whose responses are logically incon- 
sistent with one another could be determined since telephone corrected and 
completed data were also available. As discussed in Section 3.2, the ex- 
perimental data set that was constructed of the original responses to the 20 
selected critical items had. a relatively low rate of item nonresponse and a 
somewhat larger rate of inconsistencies. The rate of inconsistencies for an 
— \ item is somewhat misleading since it is a measure of the proportion oi times a 
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response given to that particular item disagrees with other responses on the 
same instrument. An inconsistent response may be correct or incorrect. 
However, if the responses to two different items logically disagree, then at 
least one of the items is incorrectly answered. This obvious source of mea- 
surement error was investigated as well as nonrespondent bias. To the extent 
that the 20 critical items are representative of the NLS instrument, comments 
concerning response and nonresponse errors in estimates and procedures to deal 
with these errors apply to othei items on the instrument as well. 

In general , considering the fact that the critical items selected were 
judged most prone to response and nonresponse errors, the quality of the data 
appears to be relatively good, even before the telephone editing. Three mea- 
sures of quality — the relative bias, the relative root mean square error, and 
^ the bias ratio of an estimate--were used to evaluate the accuracy of the esti- 
mates resulting from the 20 selected critical items. 

The relative bias (RB) is defined to be the bias divided by the popula- 
tion value being estimated. This measure of data quality reflects the common 
specification of data analysts that gueater accuracy is needed for estimates 
of smaller population values. For this study, the relative bias of a statis- 
tic was estimated as the bias (the difference between the statistic and 
Y-TRUE), divided by the "true" population value (Y-TRUE) obtained using the 
telephone edited and corrected data. Of the 51 estimates computed for the 
entire population using the experimental data set of 14 discrete items with no 
imputation or editing procedure (NI),.42 have moderate to low relative biases 
of less than 5 percent (see Table 4-1). The average relative bias for the 51 
discrete estimates was 2.25 percent. The estimates of the mean value for the 
entire population for the 6 cbntinuous questions had an average relative bias 
of 0.8 percent, and all estimates have relative biases of less than 5 percent 
(see Table 4-2). 

The relative root mean square error (RJMSE) , which was defined to be the 
square root of the mean square error of the statistic being estimated divided 
by the value of the estimate obtained when the telephone edited and corrected 
data were used (Y-TRUE), is a better measure of the quality of the data base. 
However, these estimates obtained using the experimental data set do not re- 
flect the quality of the N£S data set, since the experimental data set was 
only about three-fourths the size of the full data set. This occurred due to 
the fact that only one-hal/ of the fail-edit mail questionnaires were included 
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Table 4-1. — Comparison of no imputation estimates when inconsistent data is removed (NIC) and whenAretained (NI) - 
Part 1: Proportions estimated for the total population 



HEM 



RESPONSE 



ft 

sample! J 



rnur 





ME 


NI 


MIC 


NI 


NIC 


NI 


NIC 


BIAS 


RBS 


HBX 


RttS 


BK 


BR 


RVftSEX 




-1.7H 


-2 .HO 


-2,38 


0.96 


-H.21 


1.52 


2.H5 


1. 1H 


-0.07 


\ -0.H0 


•0.31 


-8.97 


-0 • 1H 


-H . H 0 


2.21 


9.19 


0,30 


7.28 


7.06 


-8. 37 


1 .67 


-1 #70 


8.23 


9.70 


-1.25 


-13.59 


-13.67 


-13.37 


-5.06 


-H • 9U 


13.93 


13. 6H 


-H .H9 


-6.62 


-6,69 


•0 .98 


-9. 05 


•1 # 1 3 


6.73 


1 • 30 


-0.56 


-1.7H 


•1.63 


-9.72 


-1.03 


-6.81 


2. 27 


9.03 


0.01 


0 .25 


0,53 


-16. H7 


0.11 


-H ,H3 


* H .67 


18.93 


-1.57 


-22. HI 


-22 4 5H 


-22.8H 


-5.21 


-1 .67 


22.96 


23.36 


0.06 


0 .09 


0,09- 


1 .99 


0.10 


2.28 


0.87 


2.18 


-0.05 


-0.38 


-0.2H 


-9.53 


-0.09 


-3.83 


2.66 


9.85 


-0. 06 


-H . 13 


-5,97 


-1H.93 


-0.77 


-1 #82 


9.76 


17. 02 


0.05 


0.20 


0,26 


0.98 


0.16 


0.6H 


1.62 


1.81 


1.38 


5.37 


5,03 


-12.77 


1.H1 


-3. 38 


6.16 


13.32 


-0.01 


-0.13 


0,52 


5.38 


0.07 


0.72 


7.05 


9. 17 


"1.37 


-2 .OH 


-1.98 


H. 30 


-1.18 


2.39\ 


2.59 


H .66 


0.01 


0.01 


•0,03 


0.06 


0.09 


0.15/ 


0 .39 


0 . HO 


-0.01 


-0.11 


-0.39 


-0.6H 


-0.09 


-0. ly 


H.22 


H • 32 


1.26 


7.17 


6,79 


-22.97 


1.51 


-H .52 


8. 1H 


23.53 


0.31 


8.68 


9.81 


5.95 


1.33 


0.70 


12.26 


10.33 


-1.57 


-1.99 


-1.95 


H.8H 


-1.90 


3.71 


2.20 


5.01 


0.H2 


0.89 


0.H8 


-0.23 


0.36 


-0.18 


1.H0 


1 .31 


-0 .<*,2 


-0.79 


-0.H3 


0.21 


-0.36 


0.18 


1.25 


1 • 17 


0,2*4 


0 .H8 


0.61 


-0.10 


0 .H5 


-0 . 12 


1 .H7 


1 .H7 


-0.2H 


-0 .H7 


-0.60 


0.18 


-0.H5 


0 • 12 


1.H6 


* 1.H6 


0.29 


1 .H2 


1.32 


•1.17 


0.77 


-0.59 


2. 16 


2.29 


1.08 


' 3.37 


H.15 


3.73 


2.32 


2.15 


H.52 


H. 11 


-1 .38 


-2.89 


-3.35 


-2.00 


-2.26 


•1 .50 


3.66 


2.HC 


0. 12 


0.18 


r 0.19 


0. 81 


0. 16 


0.73 


1 • 16 


1.38 


0.26 


5.57 


5.73 


•0.61 


0.77 


•0.09 


9.33 


«. 6.58 


0.02 


0 . 3H 


/ 0.93 


-1.15 


. 0.2H 


-0.18 


6.59 


6*36 


-0 ,H0 


-1.67 


-1.87 


-1.8H 


-0 .52 


-0.52 


H.03 


3.99 


-0.01 


-0 . 01 


-0.12 


-0.15 


-0 . 30 


-0. 30 


0.H2 


0.H3 


0.01 


0.06 


0.66 


0.82 


0 • 30 


0 .38 


2.26 


2. 30 


-0.53 


^0.79 


-0.H6 


0. 3H 


-0.23 


0 • 16 


2. OH 


2.08 


0.53 


1 .58 


0.92 


-0.69 


0.23 


-0 • 16 


H . 09 


H. 15 


0.00 


0.00 


-0.09 


-0 .05 


-0 .H2 


* -0.21 


0 .25 


0 . 2H 


0.01 


1.11 


1 .98 


- r • 2 3 


0.19 


-0.20 


10 • 33 


11.01 


-0.01 


-0.15 


1.16 


1 . OH 


C.29 


0.26 


H .06 


H • 00* 


0. 02 


0 .20 


-0. 17 


-0. 55 


'•O. OH 


-0. 1H 


2 • 92 


3.97 


0.00 


0 .00 


-0.06 


-0. 02 


-0. 07 


-0. 02 


0 • 91 


0.91 


r 0 , 0 1 


-0 .2H 


0.07 


0 *. 02 


0.01 


0.00 


5.08 


5.07 


-0.0| 


-0.02 


.0.10 


0.16 


0. 15 


b.22 


0.70 


0.73 


.0.11 ' 


0.15 


0.07 


0.U3 


0.08 


0.03 


0.91 


0.93 


-0.02 


-0^.25 


-0.03 


0.19 


0.00 


0.03 


6.38 


6.HH 


0.0 0 


o'.oa 


0.08 


• 0.32 


0.00 


0.02 


11.53 


11. 5H 


-0.02 


-0.07 


0.16 


0.26 


0.0H 


0.07 


. 3.38 


3. HI 


0.02 


0.10 


0.00 


0.06 


0.00 


0.01 


3.H5 


S 3.H5 


0.00 


0.00 


0.00 


-0.01 


0.00 


-0.01 


1.06 


1.06 


0.00 


0.00 


0.37 


0.39 


0.06 


0.06 


5.71 


5.72 


0.00 


0.00 


-0.19 


-0.16 


-0.02 


'0.02 


7.16 


7.76 


O.Ul 


0.0'J 


-0.39 


-0.H3 


-0.11 


-0.12 


3.52 


3.51 
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IM1A 


1 




1 


TU10 


1 


f g|G 


1 


TU9A 


1 


1 09C 




TU9U 


1 


I <J9tf 


1 


tui n 
1 **iu 


1 


IO10 


2 


mio 


3 


f U1Q 


H 


TU12 


1 


TU12 


2 


f IJ12 


3 


TU29 


1 


TU29 


2 


1033 






2 


TW33 


3 


1 TU51 


I 


I s ? TU51 


2 


1 TU52 




I«52 ' 


2 J 


TU6b 


1 


1 U66 


2 


1 W66 


3 


I W90 


1 


TO90 


2 


TU90 


3 


1090 


H 


IM101 


1 


TU10 1 


2 


f 0102 




TUlUi? 


2 


f U 1 1 5 


1 

A 


^^•w 1 Ul 18 
ful 1 M 


2 
1 


TU12* 


I 


TU129 


2 


IU12* 


3 


I U129 


H 


TU131A 


1 


TU131C 


1 


IU131U 


1 


TU1UF 


1 


TU1I6U0X 


1 


TUI lb A 


1 


I«libC 


1 


IM1/360 


1 
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15089 
15089 
15089 
15089 
15089 
15089 
15089 
15089 
15089 
15089 
15089 
15089 
36HH 
36HH 
36HH 
11H39 
11H39 
H23H 
H23H 
H23H 
15089 
15089 
7579 
7579 
7579 
7579 
7579 
7579 
7579 
7579 
'379 
15089 
15089 
2199 
2199 
15089 
15089 
15089 
15089 
15089 
15089 
15089 
6336 
6336 
6336 
6336 
7010 
57H3 
57H3 
57H3 
57H« 



72.29 
17 t 16 

H. 12 
9.19 

67.82 
32.15 
3.96 
7.00 
61.22 
13.06 

I. H5 
2H.27 
25.66 

7.20 
67. 1H 
91.H0 
8.60 
17.56 
3.57 
78.87 
H7.13 
52.87 
H9.75 
50.25 
20.29 
32.03 
H7.68 
65.6H 
H.67 
5.79 
23.90 
8H.17 
15.83 
66. 6H 
33.36 
92.68 
0.90 
6.H2 
9.78 
H5.95 
H.02 
HO. 25 
73.00 
7.87 
H.30 
26. HO 
18.19 
71.91 
tf.09 
3.89 
21.81 
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ie a-x. (continued; — Corapanson oi no imputation estimates wnen inconsistent aaca is removed itJIC) and when 

retained* (NI) - Part 2: Means estimated for the total population 



tren 


SAMPLE 


Y-THUF 


«E 
BUS 


ME 
ROX 


HI „- 
HOX 


NIC 
Rftt 


NI 

8R 


NIC 
BR 


NI 
.RtflSE* 


NIC 
fttfiSEft 


• U15 

IU«9hA 
1 U89HI1 
IU1HUA 
IU1HIFB 


1HH5 
11<*5 
7579 
?579 
15069 
15089 


36.75 
1*0. 2ft 
2102.30 
2173.09 
7039.66 
670H.H1 


0.00 
-0.21 
7.6ft 
H0«05 
-17. 3H 
-36.25 


0.00 
•0.13 ' 
0.36 
1.64 
-0.24 
* -0.H3 


0.0H 
0.1* 
1.20 
2.36 
-0.HH 
-0.57 


0.1* 
0.27 
0.60 
0.56 
-0.30 
-0.41 


0.12 
0.22 
0.67 
1.30 
-0.H*4 
-0.66 


0.H3 
0.HH 
0.«*5 
O.HH 
-0.29 
-0.53 


0.33 
0.63 
1.61 
2.66 
1.06 
1.0H 


0.37 

0. 46 

1. %5 
1.41 
1.06 
0.69 



SAMPLE SIZE - 



Y-TRUE - 
ME - 



RBX - 
BR - 

r/mse% - 



Glossary of Terms Used in Table 

number of sampie members eligible correspond to a particular item for the domain 
under consideration. 

the estimate obtained using the telephone corrected and 'completed data. 

measurement error caused by. the use of data containing logical inconsistencies. 

the relative tkas defined to be thfe bias divided by the value of Y-TRUE, 
expressed as a percentage. 

the bias ratio defined as the bias divided by the standard error of the estimate. 

t $ 
the relative root mean square error defined as the square root of the mean square 

error divided by the value of Y-TRUE and expressed as a percentage/ 




Table 4-2. — Average quality of the data for selected subdomains 









Discrete 


I tens 






Continuous Iteas 






Don;* in 


Average 
ME 

|RHZ| 


Average 
Nl 

|m| 


Average 
HI 
|BR| 


Average 

HI 
RifiSEX" 


Average 
ME 

|*b:| 


Average 
HI 

|RBZ| 


Average 
NI 
|HR| 


Average 

NI 
R*tiSEZ 


Total 




2.11 


2.25 


0.91 


4.64 


0.50 


0.80 


0.64 


1.29 


Sex: 


Hale 
Fennle 


1.98 

2.62 


2.08 
2.78 


0.65 
0.64 


5.90 
6.96 


0.72 
0.29 


1.01 
0.43 


0.50 
0.26 


1.87 
1.59 


Ability: 


Low 

Middle 
High 


• 2.63 
2.72 
2.31 


2.98 
2.83 
2.71 


0.43 
0.64 
0.59 


8. A3 
7.07 
8.62 


0.69 
0.86 
0.55 


1.42 
0.79 
0.61 


0.37 
0.35 
0.32 


3.26 
1.97 
1.80 


SES: 


Low 

Middle 
High 


2.41 
2. SI 
2.49 


2.50 
1 2.74 
2.73 


0.49 
0.66 
0.65 


7.02 
6.20 
8. 18 


0.78 
0.32 
0.72 


1.41 
0.58 
0.78 


0.54 
0.38 
0.37 


2.50 
1. 18 
1.97 


Kace : 


Black 
White 
Ulepanlc 
Other ' 


3.41 
2.00 
4.97 
2.74 


3. 74 
2.16 
3.03 
3.03 


0.49 
0.74 
0.35 
0. 35 


10.25 
5.23 
12.10 
12.10 


1.16 
0.47 
0.29 
1.04 


2.01 
0.72 
9.36 
1.19 


0.62 
0.55 
0.48 
P. 28 


4.02 
1.30 
17.28 
J. 51 


Region: 


Northeast 
Sooth 

North Central 
West 


.2.64 
2.16 
2.39 
2.32 


2.86 
2.43 
2.74 
2.53 


0.51 
0.69 
0.59 
0.51 


9.19 • 
7.11 
6.23 
8.09 


0.88 
0.38 
1.12 
0. 35 


1.28 
0.38 
1.50 
0.51 


0.49 
0.26 
0.66 
0.23 


2.54 
1.50 
2.60 
2.30 


Race x Ability: 


















Black: 


Low 

Middle 


4. IS 
3. IS 


4.96 
3.69 


0.42 
0.35 


14.76 
21.99 


1.75 
0.53 


4.72 
1.27 


0.67 
0.22 


8.15 
6.45 


White: 


Low 

Middle 
III pji 


2.42 
2. 86 
2.31 


2.73 
3.00 
2.77 


0.26 
0.58 
0.57 


12. 13 
7.82 
A. 64 


0.79 
0.A2 
0.57 


0.69 
0./2 
0.6) 


0.21 
0.31 
0.33 


3.71 
2.01 
!.?•! 


Other: 


Lou 

Middle 

iriKii 


4. 58 
3.24 
).22 


5.06 
3.68 
3.54 


0.25 
0.20 
0.22 


22. 76 
22. 12 
31.08 


1.46 
2.67 
0.59 


2.85 
2.86 
1.21 


0.33 
0.28 
0.20 


7.98 
7.25 
7.08 



NOTE. — The domain estimates tor 'blacks of high ability and cross tabula- 
tions involving Hispanics were not included in this table since there 
were too few in the sample to compute valid variance estimates. For the 
rest of the domains, the average of the absolute value of the relative 
biases expressed as a percentage (|RB%|), is given for the measurement 
error due to inconsistencies in. the data (ME) and for the total error of 
the no imputation (NI) esitmatesJ For the no imputation estimates, the 
O age of the absolute values of the bias rat ios (|BR|) and the average 
KJ^C he relative root mean square errors (R/MSE) are also given for each 



in the data set. The weights were adjusted as discussed in Section III-A to 
account for the subsampling so that the means and proportions are valid esti- 
mates; however, the variance of the estimates will be larger than what would 
be obtained h^d the full data set been used. Thus the relative root mean 
square errors of the estimates will also be larger than if the full data Set 
had been used. For the estimates of proportions using the entire population 
as the domain, 34 of the 51 estimates had relative root mean square errors of 
less than 5 percent and 46 were less than 10 percent. The average relative 
"root mean square error for the 51 proportions estimated from the 14 discrete 
items was 4.64 percent. All of the estimated means for the entire population 
resulting from the 6 continuous items had relative root mean square errors 
that were less than 5 percent, with an average value of 1.29 percent (see 
Table 4.2). 

A final measure of the quality of the survey estimates is the bias ratio 
(BR). f The bias ratio was defined to be the bias ol an estimate divided by the 
standard deviation of the estimate. The bias ratios presented in this paper 
are, again, not the same, as those which would have resulted had the full data 
set been used. In this case, the bias ratios should be smaller than those for 
-the fiitl data set since the denominator or standard deviation is larger due to 
subsampling increasing the variance of the estimates. As mentioned previous- 
ly, the error rates for confidence intervals double for every increase of 0.5 
in the absolute value of the bias ratio. Considering that the bias ratios 
obtained using the experimental data set would be expected to be less than 
those using the full data set, the quality of the estimates with respect to 
the bias ratios' is not as good. Because the large sample size results in 
small variances, even a moderate bias can have a large effect. Looking at 
total population estimates, 17 of the 51 estimated proportions and 3 of the 6 
estimated means have absolute bias ratios greater than 0.5 with 13 proportions 
having bias ratios greater than 1.0. .From examining Table 4.1, one can see 
that the large bias ratios are mainly associated with TQ1 and TQ9 and with 
items* within. routing patterns. 

To determine the effect on the bias of inconsistencies which were occur- 
ring at a high rate, a data set was constructed that contained the original 
inconsistent data but had all nonresponse replaced by the response obtained in 
the telephone interview. The difference between the estimates obtained using 
this data and the estimates obtained using the data set with both inconsisten- 
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cies and nonresponse corrected is the bias due to the presence of inconsistent 
responses which will be referred to as the measurement error (ME) bias. Note 
that the estimates obtained from the experimental data set using no imputation 
or editing (NI) contain both this measurement error bias as well as nonre- 
sponse bias. Referring to Table 4.1, which gives both of these bias terms for 
the proportions estimated for the 14 discrete items, one can see that measure- 
ment error caused by inconsistent (and incorrect) responses was the most 
important component of the bias in the no implication estimates. This is to be 
expected since the item nonresponse rate for discrete ite.as was less than two 
percent in the experimental data set; hence, the assoc'ated nonresponse bias 
would be expected to be small as well. 

A general conclusion that can be made for the discrete items is that 
nonresponse is not an important factor in the bias of the no imputation es- 
timates and any imputation procedure that mefrely replaces missing values will 
not compensate for the most important source of bias, namely, measurement 
error due to inconsistent and incorrect responses. In fact, in many instances 
the nonresponse bias and the measurement error bias have opposite effects so 
that the total bias in the no imputation estimates is less than the measure- 
ment error bias. One easily implemented approach to the problem of inconsis- 
tent data might be to remove the inconsistent responses to various items and 
code the responses as missing. This procedure was used on the experimental 
data set and estimates were obtained using no imputation on this new data set 
which had only consistent responses but many more missing responses. Refer- 
ring again to Table 4-1, which gives estimates for the discrete items for the 
entire population, one can see that, in general, the bias of the no imputation 
estimates when inconsistent data is removed (NIC, for no imputation, consis- 
tent data) is larger than the no imputation estimates obtained using the 
inconsistent data (NI). The problem is that a data item may be inconsistent 
with another data item and yet be correct, so that by discarding all responses 
to an item that are inconsistent with responses to other items on the same 
instrument, information is lost and nonresponse becomes a more serious prob- 
lem. For instance, the discrete items TQ1 and TQ9 were multiple response 
option items in which the individual was questioned concerning his activities 
in October 1976 and October 1975 respectively (see Appendix C for the text of 
the questions). Although the individual was instructed to circle all options 
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which applied to him, a typical error was for an individual to circle only 
his principal activity. For instance, an individual working full time and 
taking one or two college courses at night often did not circle option num- 
ber 3, "Taking academic courses at a two- or four-year college. 1 ' Response 
errors such as these were often detected as an inconsistency when the indivi- 
dual responded ^jfes" to TQ52 or TQ66, which asked if classes were taken during 
October 1976 and October 1975, respectively. Thus, inconsistent responses to 
TQ1 and TQ9 were often of the form where one option was circled when one or 
two additional options should have been circled as well. One can see, in this 
instance, that discarding all responses to TQ1 and TQ9 when an inconsistency 
is identified will not solve the problem. The discarded inconsistent data 
(which will now be regarded as missing) ar^e atypical of the remaining data in 
that the proper edited responses for these excluded individuals are nearly » 
always combinations of two or more of the possible activity states. Such 
deletions also exclude good response items which are inconsistent with other 
faulty items; for example, TQ52 may correctly indicate classes taken in 1976, 
while TQ1 fails to include the "Taking academic courses ..." option resulting 
in the deletion of good TQ52 data. Obviously, removing responses to items that 
are inconsistent with one another is not an adequate way of reducing the bias 
resulting from the use of data containing logical inconsistencies. 

Another concern in evaluating the importance of response error biases is 
the effect on the bias of domain estimates. The following two sections will 
evaluate response error bias and total bias for the entire population and then 
for the selected domains of males, individuals of high ability, individuals 
with low socioeconomic status (SES), individuals from the South, and blacks of 
average ability. These particular domains were selected as being of special 
interest in illustrating the effects of differing response patterns and dif- 
fering sample sizes. Tables presenting the results for these domains are given 
in Appendix F. Since removing inconsistent responses almost exclusively 
resulted in poorer estimates, only the response errors for the no imputation 
statistics computed using the experimental data set containing both inconsis- 
tent and missing data will be evaluated. 

1. Response Errors in the Estimates ior the Entire Population 

The problems created tiy response error bias are entirely different 
for estimates obtained from discrete aud continuous items, so these were 
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analyzed separately in this investigation. The differences can be seen by 
examining Attachment B-2 of Appendix B which gives the rate of nonresponse for 
each item and other response errors for the subsample of fail-edit question- 
naires. 

The rate of inconsistencies (i.e., where a response to a data item 
was inconsistent with the responses to one or more items on the same instru- 
ment) was partitioned into a rate for inconsistent but correct, and another 
rate for inconsistent and incorrect. The designations of correct and incor- 
rect were determined by comparing the original data with thfe data obtained in 
the telephone interview. The discrete items TQ1 and TQ9 which were discussed 
in the previous section have especially large rates of inconsistent and incor- 
rect responses. In addition to these two items, TQ33 an$i TQ66, which are 
discrete items located within routing patterns controlled by TQ32 and TQ51, 
respectively, haVe somewhat large rate§„of inconsistencies. For the discrete 
items, the irate of missing data was quite low, with 4.4 percent of the sub- 
sarapled fail-edit questionnaires leaving TQ118 blank, and 2.4 percent leaving 
TQ90 blank— the highest nonresponse rates found. For the continuous items, 
nonresponse was a much greater source .of error than inconsistencies. Items 
TQ141FA and TQl^lFB, which requested total income in 1975 and 1976, respec- 
tively, eact^ had nonrespon&e or bad data amounting to 23.9 percent of the 
fail-edit questionnaires. On a somewhat smaller scale, TQ89HA and TQ89HB, 
which requested the total cost to the student to live and go to school in 
1974-75 and J£75-76, respectively, had ^nonresponse rates of 4.9 percent and 
5.2 percent. Inconsistent and incorrect data occurred at the highest rates 
for TQ89HA and TQ89HB also, with 4.2 percent and 4.6 percent responding incor- 
rectly. In general, though, one may say that the items with categorical 
responses had low rates of nonresponse and higher rates of inconsistencies, 
while the items with quantitative responses had the opposite situation of 
lower rates of inconsistencies aad higher rates of nonresponse. 

The effect of the inconsistent and inctfrrect responses in the dis- 
crete items can be seen from an examination of Table 4-1. Items TQ1 %nd TQ9 
displayed the most inconsistent responses, and it is for these estimates that 
the largest values of the relative bias, bias ratios, and relative root mean 
square errors occur. For each of these two items , the proportions circling 
response options A, C, D, and G were estimated. The bias was negative, in 
general, indicating that the proportions would have beep underestimated had 
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the inconsistencies not been corrected by telephone interview. The largest 
biases were recorded for TQ1G and TQ9G in which the proportion unemployed in 
1976 and 1975 was underestimated by 14 and 23 percent, respectively, in the no 
imputation estimates. Many of the remaining large values encountered for the 
relative root mean square error were associated with the estimation of small 
proportions from subsets of the entire domain. These large values are to be 
expected since the variance of the estimates of small proportions will be 
large in relationship to the value of the estimate. Furthermore, by using 
only three-fourths of the full sample, ^the variances of these estimates will 
be greater than for the full sample. The effect on the variance of estimating 
small proportions is more pronounced when the estimation is being done for a 
subset of the entire population, suctu as those who attended school in 1974- 
1976 (the only individuals ^eligible to respond. to TQ52 - TQ97), those married 
in the first week of October 1976 (the only individuals eligible to respond to 
TQ130 - TQ136) , or those unemployed in the ■ first week • of October 1976 (the 
only individuals eligible to respond to TQ11 and TQ12) . With respect to the 
bias ratio, the largest; values appeared for items TQ1 and TQ9 and for items 
TQ12, TQ33, TQ66 and TQ90, which are items within routing patterns. 

To gain a further understanding of the effects of the response 
errors to TQ1 and TQ9, some cross tabulations were computed for the total 
domain. For purposes of comparison, cross tabulations of some similar re- 
sponses to TQ131 and TQ136 were also included. Note that TQ131 and TQ136 asks 
the respondent about his spouse 1 s activities in the first week of October 1976 
and in .October 1975, respectively, using a format similar to that used for TQ1 
and TQ9 which ask about the individual's activities. These tabulations are 
presented in Appendix G. One can easily see that the quality of those cross 
tabulations involving TQ1 and TQ9 is very poor. The relative bias for the 
estimate of the proportion of individuals who worked and took academic courses 
at a 2- or 4-year college in October 1975 (TQ9A and TQ9C both circled) was 
-24.70 percent. The bias ratio for this estimate was -14.76. Note that the 
cross tabulations for items TQ131 and TQ136 did not display this bias due to 
inconsistencies. However, for these two items, there were no similar items on 
the questionnaire that could be used for editing of logical inconsistencies. 

In general, one can see the need for editing of the responses to TQ1 
and TQ9 since the individual tabulations and cross tabulations involving these 
items were of poor quality. Because of the importance of these two items in 



classifying an individual's work and academic activities, NLS resolved all 

inconsistencies using a telephone interview. It may be argued that the data 
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in TQ131 and TQ136 should be used with caution since their format is similar 
to TQ1 and TQ9. The instrument did not provide a means for checking these 
items with respect to consistency so that faulty data could not be detected 
and corrected in the telephone interview. 

The quality of the. estimates obtained from the continuous data was 
better than that of the discrete data even tbougt some of the items had large 
rates of nonresponse (see Table 4-1). The relative bias caused by inconsis- 
tencies was quite low which contributed to the fact that the no imputation 
estimates had reasonably small relative biases and relative root mean square 
errors. Nonresponse bias, even for TQ141 which had nonresponse amounting to 
around 13 percent , • did not have a significant effect on the quality of the 
estimates' for the total population. However, the bias ratio for 3 of the 6 
means exceeded 0.50. The significance of this with respect to the use and 
interpretation of confidence intervals was discussed previously. The biases, 
though small in relation to the value being estimated, are large with respect 
to the standard deviation of the estimates . Thus, the total bias of the esti- 
mates is of concern and some procedures are necessary to correct for it. 
2. Response Errors for Domains 

As may be seen from an examination of Table 4-2 and the tables in 
Appendix F, the patterns of response errors were basically the same for the 
domains composed of males, individuals of high quality, individuals with low 
socio-economic status (SES) , ^individuals from the South, anil blacks of average 
ability as the response error pattern was for the total population. A compar- 
ison of these tables should be made carefully as domain sample size has an 
effect on the bias ratio and relative root mean square error. For instance, 
the bias "ratio of estimates for males may be expected to be lower than for the 
total population since the standard deviations of the estimates for males 
(which forms th^ denominator of the ratio) * are larger than the comparable 
standard deviations of the estimates for the entire population. Thus, the 
smaller sample size of domains will result in a larger variance which will 
cause the bias ratio to decrease and the relative root mean square error to 
increase with respect to that of the total population. In comparing estimates 



for domains of different sizes and in comparing estimates for different items 
within the same domain (not all domain members answer each item because of the 
preseace of routing patterns), the bias ratio and root mean square error 
should be used -(Ph cautiously because of the effect due to sample size. The 
remainder of this section discusses various aspects of response errors using 
examples from domains. 

The comparison between the quality of data for individuals of high 
ability and the entire population gives unexpected results. One's intuition 
might suggest that high ability individuals would have less trouble than the 
general population in understanding and dCsponding to the instructions in the 
instrument. Referring back to Table 4-2, one antes that the average of the 
absolute relative measurement error biases for the discrete items is 2.31 
percent for high ability individuals as opposed to, 2. 11 percent for the entire 
population. Similarly, for the continuous items^ the average of the absolute 
relative measurement error biases is 0.55 percent for the high ability indi- 
viduals as opposed to 0.50 percent for the entire population. Since the item 
response rates were essentially equal for the two 'domains, it is likely that 
the difference in the average quality of the data with respect to relative 
bias can be attributed to measurement error rather than nonresponse. 

An exceptiou to this result occurs for Item TQ118. * A large increase 
to -13.67 percent wa.> seen in the .relative bias for the no imputation estimate 
of the proportion of high ability individuals responding "2" to TQ118 (indi- 
cating that since October 1974 they had served in the National Guard or Re- 
serves) . Since the experimental data set had no inconsistent, incorrect 
responses to this item, this bias was due entirely to nonresponse and illus- 
trates how even a small rate of item nonresponse (approximately 2 percent of 
the full NLS data set) can have a serious effect when a small proportion (in 
this case 0.41 percent) is being estimated. 

As may be seen in Table 4-2, the average quality of the data for 
blacks tended to be somewhat worse than that observed for the entire popula- 
tion. Much of the difficulty appears to be in the interpretation of routing 
patterns where large measurement error relative biases were observed (28 .57 
percent for response 2 of TQ90 and 18.60 percent for response 2 of TQ33 as 
seen in tables presented in Appendix F) . These larger measurement error 
biases lead to larger values for the relative bias of the no imputation esti- 
mates and contribute to the larger values of the relative root mean square 
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error. The effect of bias in conjunction with a small size can be seen in the. 
relative root mean square errors for the discrete estimates where only 16 of 
the 51 estimates' have relative root mean square errors of 5 percent or less; 
whereas 34 of the estimates for the total population have relative root mean 
square errors of 5 percent or less. 

Two of the more striking examples of the effect of measurement error 
bias occurred for the estimates of the proportions of blacks of average abil- 
ity circling IQ1G, indicating they were unemployed in October 1976 and October 
1975, respectively (see Appendix F). For these proportions, the relative 
biases of the no imputation estimates were -27.42 and 46.85 percent, respec- 
tively. If telephone editing had not been used to correct t'EeSnconsistencies 
for these items, the proportion of black individuals unemploye^in October 



f976 and October 1975 would have been estimated as 11.32 and S.^S^percent 
instead of the 15.60 and 11.23 percent estimates obtained after editing. 

The examples given above illustrate the effect that measurement 
error and nonresponse bias can have in the presence of samples of different 
sizes. The most surprising aspect of the examination of response errors by 
domain was that there is little variation in the pattern cf errors across 
domain. Minority groups such as blacks and Hispanics tended to have data of 
lesser quality with respect to relative bias than the general population, but 
these differences were not major (see Table 4-2). . The overall quality of the 
data with respect to relative root mean square error was rather low for many 
domains, but this was predominately a function of* small sample sizes. 

B. Comparison of the Performance of Hot Deck and Weighting Class 
Estimators with That of No Imputation Estimates 

It l ks clear from the previous discussion that for discrete items in 
general and for the continuous items within routing patterns, logical in; 
consistencies occurring in the data can have a serious effect on the bias. 
For instance, if all missing data were replaced by the true value using either 
an imputation procedure or telephone foliow-up, the measurement error bias 
associated with an individual failing to respond in a* correct manner to the 
questions would still be present and cculd cause serious problems in inter- 
preting the estimates obtained from some of the items . The focus of rhis 
investigation was to investigate nonresponse bias so the potential benefits of 
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logical editing to correct inconsistent data were not investigated. However, 
the hot deck and weighting class imputation procedures did force the response 
within a routing pattern to agree with the lead-in question. When an in-" 
consistent response (or responses )xtfas encountered within a routing pattern, 
the entire set of responses to the items within the routing pattern were 
replaced (see Sections C.2 and C.3 for further discussion). Thus, the perfor- 
mance of the two imputation procedures depends upon three factors: the amount 
of nonresponse bias in the estimates, the extent to which the imputation 
procedure reduces the nonresponse bias , and the dey "ee to which replacing 
inconsistent responses within a routing pattern imptv..es the quality of the 
estimates. These issues will be discussed with respect to the discrete items 
as a group and then with respect to the continuous items. 

Tables presenting the relative bias (RB) , the bias ratio (BR) , and the 
relative root mean sjuaj;e error (RVHSE) of the no imputation (NI), hot deck 
(HD), and weighting class (WC) estimates for all items may be found in Appen- ^ 
dix H, for these selected domains - total population, males, high ability 
individuals, blacks, individuals of low socioeconomic status, Southerners, and 
blacks of average ability.. Further, Appendix I gives the RB, BR, and R>/HSE of 
these estimates for all domains for selected discrete items and for all of the 
continuous items. 

1. The Success of the Nonresponse Imputation Procedures in Reducing the 
m Total Bias of Estimates Associated with Discrete Items 
An examination of these tables given in appendices H and I reveals 
that .the weighting class imputation procedure devised for use with discrete . 
items almost uniformly produced estimates whose total error was greater than 
that of the hot deck and no imputation estimates. Part of the reason the 
weighting class imputation technique performed so poorly was that the number 
of nonrespondents to an item within each weighting class was so small (often 
aflBMk as one or two individuals) that it was impossible to have the propor- 
tiSrof nonrespondents assigned each response equal the proportion o r respon- 
dents who gave the response. Because of its obvious lack of efficacy in 
reducing the total error of the discrete items, the weighting class imputation 
procedure was not subjected to a lengthy analysis to compare the estimates 
with those produced when no reputation or editing procedure was used. The 
results of the use of the weighting class procedure are s'unmarized in Table 
A- 3, however. 
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In general, the hot deck imputation procedure did appear to have 
some effect iii reducing the bias of survey estimates* Table 4-3 gives the 
averages for ail 51 estimated proportions fcf the absolute relative bias, the 
absolute bias ratio, and the relative root mean square errors by selected do- 
mains of interest. For almost all of these domains, the average absolute 
relative bias and the average absolute bias ratio are less for the hot deck 
estimates than for the no imputation estimates. For instance,, the average 
absolute relative bias of the estimates for the total population is 1.92 per- 
cent as coiipa red "with 2.25 percent when no imputation is used. Similarly, the- 
average absolute bias ratio is 0.83 for the hot deck estimates as compared 
with 0.91 for the no imputation estimates. The bias ratio would be expected 
to be less since the, variance of the hot deck estimates is usually greater 
than that of the no imputation estimates reflecting the greater variability of 
the hct. deck procedure. Hov/ever, the fact that the average relative bias is 
smaller does indicate that over all the 51 estimates, the hot deck procedure 
is reducing the bias in general. The domains in which the hot deck estimates 
are not better on the average, tend to be the small domains such as race equal 
to Hispanic or Dther. For those domains in which the hot deck estimates per- 
formed better, the average relative bias reduction was less than 0.5 percent. 
Much of the bias reduction resulting' from the use of the hot deck technique 
was associated with a corresponding increase in the variance* so that the 
difference between the average values of the relative root mean square error 
for the two procedures is. usually small (a tenth of one percent or less). 
Nine of the 25 domains had the average relative root mean square error less 
for the hot deck estimates, including the estimates based upon the total pop- 
ulation where the average of the relative root mean square errors was 4.53 
percent as opposed to 4.64 percent fcr the no imputation estimates. 

Another way of comparing the hot deck procedure to the no imputation 
procedure is to count the number out of the 51 proportions in which the hot 
deck estimate has a smaller absolute relative bias or a smaller relative root 
mean square error. Table 4-3 presents, this data with the number out of the 51 
estimates for which the hot deck estimates have smaller absolute relative 
biases—abbreviated as NUMBER HD | RB | < NI | RB | and the number which have 
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Sex: 


Hale 


2.08 


1.82 


2.67 


29 
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0.65 


0.58 


0.70 


5.90 


5.88 


6.24 


17 
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A 
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6.89 
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2.98 
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5.56 


23 


14 
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O.os 
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26 
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2.41 
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24 
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Low 


4.96 


4. 36 


5.95 


21 


21 


0.42 


• 0.38 


0.47 


14.76 


14.52 


15.76 
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Middle 


3.6<> 


4.16 


5.47 


18 




0.14 


0.40 


10.42 


21.99 


22.59 


26. 72 


16 
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1.4 IV 


2.73 


2. 18* 


4.78 


25 


11 


0.26 


0.29 


0.43 


12.13 


12.18 
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21 
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3.00 


2.57 


3.48 


28 ' 


15 


0.58 


•0.52 


0.61 


7.82 


7.74 


0. 21 


22 
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2.77 


2.35 


1.06 


11 


20 


0.57 


0.50 


0.5/ 
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11 
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5 0b 


7.34 


12.01 


16 


19 - 


0.25 


0.34 


0.40 


22.76 


fc 24.48 


29.78 


24 
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- 3.68 


3.10' 


4.18 


12 


7 


0.20 


0.20 


0.24 


22.12 


22.91 


23.68 
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3.54 


4.17 


4.21 


15 


17 


0.22 


0.25 


0.25 


31.08 


31.50 


31.50 


12 


10 



NOTE. --The domain estimate.- for blacks of high ability and cross tabulations involving Hispanics 
were not included" in th.o table since there were too .few in the sample to compute valid 
variance estimates. For the rest of the domains, the average of the absolute values of 
the relative biases expressed as a percentage (|RB%|), the average of the absolute values 
pf the bias ratios (|BR|), and the average of the. relative root, mean square errors (RvWSE) 
are given. The acronym "Number HD|rb| < Nl |RB| " refers to the number of estimates for which 
the absolute relative bias is- less for hot deck (HD) than for no imputation (Nl) out of the 
total of 51 estimates. Similarly, "Number HD R/msE < Nl r/mSE" refers to the number of esti- 
mates for which the r-lative root mean square error of the hot deck (HD) estimate is less 
than that of no imputation (Nl) estimate. The acronyms "Number WC |RB| < Nl |RB|" and "Number 
WC R/MSEZ" < Nl R^1SE%" have similar definitions with" respect to weighting class (WC) and 
no imputation* estimates. . 
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smaller relative root mean square errors--abbre\ ,ted as NUMBER HD RVMSE < 
NI R^HSE. For the entire population, 34 of the 51 hot deck estimates had 
smaller absolute relative biases than the corresponding no imputation esti- 
mate; whereas only 22 of the hot deck estimates had smaller relative root mean 
square errors. 

In the sense of relative bias it is interesting to note that for only 
eight of the 25 domains were more than .h^lf of the .Hot deck estimates better 
than the no imputation estimates^; Stripe Tor 20 of the 25 domains the hot deck 
estimates on the average were »<fe¥tt£p with respect to relative bias, one can 
hypothesize that the hot deck^rocedure is making substantial gains in bias' 
reduction for a few items where there is the most nonresponse; and/or that the 
procedure by which the hot deck removes inconsistent responses within routing 
patterns and replaces them with consistent responses is effectively reducing 
the measurement error associated with inconsistent responses within routing 
patterns. Nonresponse bias reduction is not likely to be the principal factor 
since the item nonresponse rate was low for all of the discrete items. How- 
ever, for the discrete item displaying the most nonresponse— TQ1 18 with ap- 
proximately two percent nonresponse — the hot deck procedure obtained less 
biased estimates of the proportions circling eac* of the three possible re- 
sponses (see Appendix I), ff one examines the average of the absolute rela- 
tive biases and relative root mean square errors on an item by item basis (see 
Table 4-4), then one carf see that th» hot deck procedure is achieving most of 
its gains on the questions within routing patterns. Of the within routing 
pattern items; ,TQ1?, TQ33, TQ52, TQ66, TQ90, TQ102, .TQ131, and TQ136; items 
TQ90 and TQ136 are the only ones in which the hot deck estimates do not have a 
smaller absolujte relative bias. Again, much of this bias reduction is compen- 
sated for by an associated increase in # the variance so that the hot deck 

estimates for TQ66, TQ102, and TQ131 have larger relative root mean squares 

✓ 

than do the no imputation estimates. 

2. The Success of the Nonresponse Imputation irocedures in Reducing 
The Total Bias of, Estimates Associated with Continuous Items 
The average values over all six continuous items of the ab.«^lute 
relative bias, the absolute bias ratio, and the relative root mean square 
error are given in Table 4-5 for the domains studied in th-'s investigation. 
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Table 4-4. —A comparison of the hot deck (HD) and no impu- 
tation (NI) estimates on an item by item basis 



Average of the absolute 
relative biases 



Average of the relative 
root mean square errors 





NI 


HD 


NI 


HD 


1 


5.86 


5.84 


6.71 


6.70 


9 


7.85 


7.87 


9.16 


9.22 


ib 


1.65 


1.79 


3.73 


/ 3.93 


12 


2.52 


1.67 


5.27 


5.23 


29 


0.22 


0.53 


2.32 


2.30 


33 


6.19 


1.74 


7.54 


5.08 


51 


0.46 


0.66 


1.34 


1.42 


52 


0.62 


0.31 


1.47 


1.14 


66 


2.95 


' 2.93 


3.45 


3.66 


90 


2.19 


2.58 


5.29 


5.38 


101 


. 0.39 


0.10 


1.35 


1.31 


102 


0.69 


0.67 


3.07 


3.15 


118 


1.08 


0.14 


4.89 


4.91 


129 


0.10 


0.12 


2.66 


2.67 


131 


0.09 


0.17 


5.56 


5.61 


136 


| 0.19 


0.13 


4.30 


4.32 



\ 
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Table 4-5. — Comparison of the average performance of hot deck and weighting class 

estimates with that of the no imputation estimates for continuous items 





Dora in 


Aver3ge 
Nl 

|rbz| 


Average 
HD 

|rbz| 


Average 
WC 

|RB£| 


Average 
Nl 

|br| 


Average 
HO 

|br| 


Average 
VC 

|br| 


Average 
Nl 
R^ilSFZ 


Average 
HD 

_ r 

RvTISEZ 


Average 
WC 
R^SKZ 


Total 




0.80 


0.76 


0.86 


0.64 


0.46 


0.64 


1.29 


1.46 


1.40 


Sex: 


Hale 


1.01 


0.94 


1.15 


0.50 


0.39 


0.53 


1.87 


7.05 


2.03 




Female 


'0.43 


0.h5 


0.61 


0.26 


0.13 


0.42 


1.59 


1.84 


1.69 


Ability 


: Low 


1.42 


2.02 


1.92 


0.37 


0.36 


0.55 


3.26 


5.41 


3.38 




U 1 A A 1 a 

n l oo i e 


n 7Q 


0. 90 


1.18 


0. 15 


0. 32 


0.41 


1.97 


2.02 


2. 19 




High 


0.61 


0.61 


0.51 


0.32 


0.31 


'MO 


1.80 


1.84 


1.72 


SKS: 


/ 

Lov\ 


1.41 


I. 55 


1.50 


0.54 


0.59 


0.57 


2.50 


2 82 


2.53 




Middle 


0.58 


0.52 


0.85 


0.38 


0.26 


0.54 


1.38 


1.79 


1.57 




llWh 


0. 78 


0. 76 


0.88 


0. 37 


0. 34 


0.43 


1.97 


2. 14 


2.00 


Race: 


f Black 


Z.01 


1.98 


2.04 


0.62 


0.40 


0.41 


4.02 


4.43 


4.06 




White 


0.72 


0.61 


0.72 


0.55 


0.13 


0.50 


1.30 


1.49 


1.36 




Hispanic 


9.36 


7. 75 


8.90 


0.48 


0.40 


0.65 


17.28 


17. 12 






Ot her 


1.19 


1.97 


1.43 


0.28 * 


0.54 


0.33 


* 1.51 


1.84 


^*3.58 


Region: 


Northeast 


1.28 


0. 74 


0.87 


0.49 


0.28 


0.35 


2.54 


2.48 


2. 33 


South 


0.38 


0.65 


0.54 


0.26 


0.38 


0.34 


1.50 


1.67 


1.51 




North fpntral 


1. 50 


2.06 


1. 76 


0.66 


0.74 


0.68 


2.60 


1.11 


2.80 




West 


0.51 


0.35 


0.73 


0.2) 


0.08 


0.11 


2.30 


3.33 


2.23 


Race x 


Ability: 




















Blmk 


Low 


4.72 


5.42 


4.80 


0.67 


0.67 


0.71 


8.15 


8.99 


7.77 




Middle 


1.27 


0.92 


2.20 


0.22 


0. 14 


0. 39 


6.45 


6.61 


f,.47 


White: 


IjOW 


0.69 


1.15 


0.69 


0.21 


0. 31 


0.30 


3.71 


7.49 


3.41 




Middle 


0.72 


0.79 


1.09 


0.31 


0.25 


0.35 


2.01 


2.09 


2.17 




High 


0.63 


0.58 


0.48 


0.33 


0.29 


0.24 


1.79 


1.81 


1.71 


Other: 


Low 


2.85 


1.7«> 


2.47 


0.31 


0.52 


0. 10 


7.98 


8.16 


7.20 




Middle 


2.86 


2.78 


2.64 


0. 28 


0.30 


0.28 


7.25 


7.26 


7.14 




High 


1.21 


1.65 


1.54 


0.20 


0.29 


0.28 


7.08 


7.74 


7.41 



NOTE. --The domain estimates for blacks of high ability and cross tabula- 
tion involving Hispanics were not included in this table since there 
were too few in the sample to compute valid variance estimates. For the 
rest of the domain, the average of the absolute values of the relative 
biases ( | RB% | ) , the average of the absolute values of the bias ratios 
(|BR|), and the average of the relative root mean square errors are given 
for the no imputation (Nl), hot deck (HI)), and weighting class (WC) esti- 
mates . 



The hot deck and no imputation estimates are more or less comparable with 
respect to the absolute relative bias, with the hot deck estimates having a 
slightly smaller average relative bias for many of the domains, including the 
total. For many domains, the weighting class estimates have slightly larger 
average relative biases than both the no imputation and the hot deck esti- 
mates. The average bias ratio for the no imputation estimates is greater in 
general than that found for the hot deck and weighting class estimates. This 
reflects the greater variability of the hot deck and weighting class procedure 
which results in a larger denominator for the ratio. The final measure of 
quality of the data--the average value of the relative root mean square 
errors--is smallest for the no imputation estimates for 16 of the 25 domains, 
including the total. It is interesting to note that although the weighting 
class estimates did not exhibit the bias reduction potential of the hot deck 
estimates, the average relative root mean square error was less for the weigh- 
ting class estimates as compared with the hot deck estimates for 23 of the 25 
domains studied. In comparison with the no imputation estimates, the weight- 
ing class estimates did better with respect to average bias and average rela- 
tive root mean square error for the smaller domains. 

The six continuous items over which averages were taken were quite 
diverse and the imputation procedures were different for those which were 
within routing patterns. To gain a futher understanding of the effect of 
using the imputation procedures on continuous data, the estimates from these 
items were evaluated separately. Tables giving the domain estimates and their 
relative bias, bias ratio, and relative root mean square error for each of the 
six continuous items may be found in Appendix I. 

The data items, TQ15 and TQ16, were related items nested within the 
routing pattern controlled by TQ10. These items requested the hours worked 
per rfeek and the average weekly salary in October 1976, respectively. Neither 

r 

the hot deck nor the weighting class procedure performed in a superior manner 
with respect to the no imputation estimates. Over all domains, the average 
absolute relative bias for TQ15 was 0.11 percent for the no imputation esti- 
mates as compared to 0.46 percent for the hot deck estimates and 0.28 percent 
for the weighting class estimates. Similarly, the no imputation estimates 
were better with respect to the average relative root mean square error with 
1.16 percent as opposed to 1.37 percent for the hot deck estimates and 1.27 
percent for the weighting class estimates. The same pattern of results was 
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observed for TQ16. The average absolute relative bias over all 25 domains 
studied was 0,23 percent for the no imputation estimates as compared with 0.46 
percent for the hot deck estimates and 0.27 percent for the weighting class 
estimates. The average relative root mean square errors were 2.29 percent for 
the no imputation estimates and 2.38 and 2.33 percent, respectively, for the 
hot deck and weighting class estimates. Note that for both TQ15 and TQ16, the 
weighting class estimates had smaller absolute relative biases and smaller v 
relative root mean square errors on the average than did the hot deck esti- 
mates . 

Items TQ89HA and TQ89HB requested the individual to estimate the 
total amount it cost him to live and go to school in 1974-75 and 1975-1976, 
respectively. These items were nested within the routing pattern controlled 
by TQ51. Even though the only difference between these two items is the time 
period being referenced, the results of the comparison of the hot deck and 
weighting class estimates with the no imputation estimates were different for 
these items. For TQ89HA, the smallest relative bias for. the total population 
was that of the hot deck estimate with 0.88 percent as opposed to 1.20 percent 
for the no imputation estimate and 1.37 percent for the weighting class esti- 
mate. Also, the relative root mean square error at 1.79 percent was less for 
the hot deck estimate for the total populatic than the no imputation estimate 
(1^81 percent) and the weighting class estimate (1.98 percent). For TQ89HA, 
the smallest relative bias for the total estimate was the 2.38 percent of the 
no imputation estimate as compared with 2.59 percent for the hot deck estimate 
and 3.04 percent for the weighting class estimate, i'he no imputation estimate 
of the total also had the smallest value of the relative root mean square 
error at 2.86 percent as compared with 3.60 percent for the hot deck estimate 
and 3. 44 percent for the weighting class estimate. 

With respect to the average over all 25 domains included in this 
investigation, slightly different results are noticed. For TQ89HA, the hot 
deck estimates had the least value for the average absolute relative bias with 
2.70 percent, compared to 3.33' percent for the no imputation estimates and 
3.14 percent for the weighting class estimates. For TQ89HB, the no imputation 
estimates had the least value for the average absolute relative bias with 3.78 
percent as compared to 4.27 percent for the hot deck estimates and 4.49 for 
the weighting class estimates . With respect to the average value of the 
relative root mean square error, the weighting class estimates were better for 
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TQ89HA with an average of 5.55 percent over all domains; whereas, the no impu- 
tation estimates averaged 5.93 percent and the hot deck estimates averaged 
6.04 percent. For TQ89HB, the no imputation estimates had the smallest aver- 
age relative root mean square error df 7.26 percent as opposed to 9.22 percent 
for the hot deck estimates and 7.39 percent for the weighting class estimates. 
Note that for both TQ89HA and TQ89HB as with TQ15 and TQ16, the weighting 
class estimates had smaller relative root mean square errors on the average 
than did the hct deck estimates. 

The final two continuous items included in the investigation were 
TQ141FA and TQ141FB which requested the total income of the individual and his 
spouse in 1975 and 1976, respectively. These two items were the only continu- 
ous items included in this study which did not fall within a routing pattern. 
Whereas the interpretation of the results for the four items within routing 
patterns was not particularly clear, the results for these two items clearly 
shows the weighting class estimates superior to the no imputation and uot deck 
estimates with respect to their relative bias and with respect to their rela- 
tive root mean square error. From examining the table for TQ141FA in Appen- 
dix I, one can see that for 16 of the 25 domain estimates the weighting class 

y 

estimates have smaller absolute relative biases than the no imputation esti- 
mates; however, only 9 of the hot deck estimates are better than the corres- 
pondiag no imputation estimate with respect to the relative bias. For 
TQ141FB, 17 of the 25 weighting class domain estimates are superior to the 
corresponding no imputation estimates; whereas only 12 of the hot deck esti- 
mates are superior to the no imputation estimate with respect to the relative 
bias. For TQ141FA, the weighting class estimates are also better in terms of 
the average of the absolute values of the relative biases with 1.27 percent as 
compared to 1.48 percent and 1.49 percent for the no imputation and hot deck 
estimates, respectively. For TQ141FB, the no imputation estimates had the 
smallest average at 0.64 percent for the absolute relative bias, followed by 
the weighting class estimates at 0.72 percent and the hot deck estimates at 
0.84 percent. The best measure of the quality of the estimates is the rela- 
tive root mean square error. It is with respect to this quantity that the 
weighting class estimates are best for both items. For both totals and for 19 
of the 24 domain estimates for TQ141FA and for 20 domain estimates for 
TQ141FB, the weighting class estimates have smaller relative root mean square 
errors than the no imputation estimates. For the hot deck procedure, six of 
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the domain estimates for TQ141FA and the total and four other domain estimates 
for TQ141FB are better than the no imputation estimates with respect "to the 
relative root mean square error. Over all 25 domains, the average relative 
root mean square error for TQ141FA was 3.43 percent for the no imputation 
estimates and 3.95 .percent tor the hot deck estimates. For TQ141FB, the 
weighting class estimates had an average relative root mean square error of 
2.73 percent; the average for the no imputation estimates was 2.90 percent; 
and for the hot deck estimates, 3.18 percent. 

It is interesting that the weighting class estimates were clearly 
superior to the no imputation estimates for TQ141FA and TQ141FB but not for 
the other four continuous items (which were different in that they were found 
within routing patterns). ^However, the weighting class estimates for all six 
items were better than the hot deck items with respect to the average relative 
root mean square error, suggesting that the weighting class imputation pro- 
cedure is more suitable as an imputation device when means are being esti- 
mated. For the items within routing patterns, one might hypothesize that the 
weighting class estimates would have been . better or comparable to the no 
imputation estimates if the procedure had been implemented differently for the 
routing pattern when one or more responses were in logical disagreement to the 
lead-in question. A check of thi's procedure for two very small weighting 
classes in the data base revealed that much good data .was being discarded by 
this~ procedure. Furthermore, the quality of the weighting class estimates of 
the means for continuous items within routing patterns would be expected to 
suffer because of the poor quality of the adaptation' of the weighting class 
procedure used for discrete data. When a routing pattern had a missing re- 
sponse for the lead-in question (which ,was always an item with categorical or 
discrete responses), the response to the lead-in question was first imputed 
and then responses to the items within the routing pattern were imputed based 
upon the imputed response to the lead-in item. Since the weighting class pro- 
cedure as applied to discrete items was .nore biased, one would expect that 
this bias for the responses to lead-in items would also have an effect on the 
bias of the estimates for continuous items within the routing pattern, 
3. Comparison of Balance Repeated Replication and Taylor Series 
Linearization Variance Estimates 

To determine the effect of ignoring imputation in computing variance 
estimates, two sets of variance estimates were computed for each set of survey 
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estimates. First, the Taylor Steries linearization estimate for the variance 
of a ratio was computed using S\TDERR,' an RTI package (see Appendix E for 
variance estimation formulas). This package calculates the standard approxi- 
mation for the variance of a ratio in terms of the variance-covariance matrix 
of the numerator and denominator totals. Second, an estimate of the variance 
of the ratio estimate was obtained using the balanced repeated replication 
(BRR) technique which allows one to account for the variability induced by the 
imputation of missing data (see Appendix D) . 

For the no imputation estimates obtained using the full experimental 
data set (NI estimates) and the experimental data set with all ij~onsistent 
data removed (NIC estimates), the two variance estimates should be equal since % 
no imputation is occurring and hence, they are both measuring the same vari- 
ability. The variance estimates for the NI and NIC estimates are given in 
Appendix J for the estimates from the following domains: total population, 
males, individuals of high ability, individuals with low socioeconomic status, 
blacks, Southerners, and blacks of average ability. To compare the two sets 
of variance estimates, the ratio of the standard deviation of the estimate 
obtained using STDERR over the standard deviation obtained using BRR (SD 
Ratio) was also computed for the tables in Appendix J. An examination of tae 
SD Ratios for the 51 discrete estimates and for the 6 continuous estimates 
indicates that the ratio is essentially one except for sampling variation. To 
see this more clearly, Table 4-6 presents the average SD Ratio for both the 
discrete and continuous items for each of the seven domains represented in 
Appendix J. The ratio of the standard deviations is more variable for the 
continuous items, reflecting the fact that only six estimates are being aver- 
aged for each domain. At 1.29, the SD Ratio for individuals of low socioeco- 
nomic status tor the continuous NIC data is the largest ratio found, ^ 

It is interesting that almost all of the average values of the 
ratios of the standard deviations are greater than on,e. Since STDERR, unlike 
the BRR procedure, allows the use of finite correction factors at the school 
level, which can be significantly less than unity, one could expect the STDERR 
estimate to be smaller than the BRR estimate and hence to obtain SD Ratios 
less than one. The fact that this did not occur is indicative that the varia- 
bility of the estimates is chiefly the result of within school rather than 
between school variation. In this case, the use of finite correction factors 
at the school level would have no important effect. 




Table 4-6. — The average value of the ratio of the STDERR standard 
deviation to the BRR standard deviation (SI> Ratio) for 
the no imputation estimates when inconsistent data is 
retained (NI) and when removed (NIC) 



Domain 




Average of 


the oD Katio 




Discrete items 
NI NIC 


Continuous 
. NI 


items 
NIC 


Total population 


1.03 


1.01 


1,01 


1.01 


Males 


, 1.09 


1.07 


0.94 


1.01 


Individuals of high 
ability 


1.07 


• 1.07 


1.06 ■ 


( 

\ 

v 1.18 


Individuals with low 
socio-economic 
status 


1.09 


1.07 


• 1.16 


1.29 


Blacks 


1.03 


1.03 


i.oi 


1.05 


Individuals from the 
South 


1.04 


1.03 


1.18 * 


1.13 


Blacks of average 
ability 


1.00 


1.00 


0.97 


0.95 



% 



62 

ERIC ' -49- 



t 

4. The Effect Ignoring Imputation has on Variance Estimation 

Typically, most users of .the hot deck and other imputation proce- 
dures ignore the fact that missing values have'^en imputed and use a standard 
software package which cannot distinguish between imputed and tjaturally occur- 
ring data in computing the variance estimates. This has the effect of magni- 
fying the real sample' size and ignoring the additional variability that the 
imputation procedure adds to the estimate. If one had a sample of 10,000 
individuals and imputed data for 10 percent who were nonrespondents to an 
item, for instance, the estimate is actually being computed from the responses 
of the 9,000 individuals who answered the item. Ignoring the fact that im- 
putation had occurred would result in an estimate obtained as if all 10,000 
individuals had responded to the item. Also, the imputation of values for the 
1,000 missing responses adds to the variability of the estimate, and the 
standard procedures cannot estimate this added variability. 

Thus, one of the goals of this investigation was to determine what 
effect on the variance estimate would result from ignoring the fact that impu- 
tation of missing responses occurred. To do this, the variances of the survey 
estimates were computed using STDERR. The variance estimates obtained using 
STDERR do not reflect the added variability induced by the use of the imputa- 
tion procedure. To obtain estimates of the variance of survey statistics 
which accounted for the added variability induced by imputation, a balanced 
repeated replication procedure was used in which the imputation procedure was 
applied tp eacfcT independent half sample (see Appendix D). The comparison qj 
these two variance estimates provides some insight into the effect on the 
variance of ignoring imputation. As can be seen from an examination of the 
tables in Appendix J which compare the STDERR and BRR variance estimators - for 
continuous and discrete items when no imputation occurred, the STDERR and BRR 
variance estimates do not track extremely well. While the average of the 
ratio of the STDERR standard deviation to that of BRR is approximately one for 
the discrete and continuous items for these selected domains, there is con- 
siderable variation about the average. This variation obscures the quan- 
tification of the effect of ignoring imputation on variance estimates since 
the variance ignoring the imputation was^ estimated using STDERR and the var- 
iance estimate which accounted for the fact that imputation occurred was 
obtained by using BRR. 

However, one can see some general effects that result from ignoring 
imputation. Table 4-7 gives 'the average of the ratio of the standard devia- 
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tions obtained using STDERR versus BRR for the 51 discrete estimates obtained 
when the # hot deck procedure is used, and the ratios for the 6 continuous 
estimates obtained using both weighting class and hot deck. From examining 
the average ratio for the estimates from the discrete items, one can see that 
the ratio is essentially equal to one. Since the nonresponse rates for these 
items was very low, the imputation would not be expected to have a substantial, 
impact on the variance. For the continuous estimates, a larger rate of nonre- 
sponse^and hence, more missing responses needed to have values imputed. The 
average ratios, are less than one in many instances although perhaps not at a 
significant level. However, the response rates were fairly high except for 
TQ141FA and TQ141FB, so ^hat little imputation would be occurring and the 
standard deviations would be expected to be approximately equal. 

One can get a better understanding of the effect of ignoring imputa- 
tion* by comparing the standard errors for the 25 domains studied in this in- 
vestigation when the hot deck and weighting class procedures are applied to 
TQ141FA, an income item which had a 13 percent nonresponse rate (see Table 
4-8). The standard deviation estimated by STDERR using the usual formula for 
the variance of a ratio is not accounting for the variability induced by the 
imputation procedure. The differential nonresponse rates within domains 
results in varying values for the ratio of the standard deviations, but they 
are usually less than one. Some of the ratios are inexplicably large such as 
the 'ratio of 1.47 for "blacks of low ability obtained when the weighting class 
procedure was used. Note that the standard deviations obtained using STDERR 
and BRR for the weighting class estimates are more nearly equal than those 
obtained using the hot deck procedure. This reflects the^ f act that the weight- 
ing class procedure induces less variation through imputation than does the 
hot deck procedure. 

. To summarize, when the nonresponse rate is large, the variance of 
sample means is underestimated when imputation is ignored in computing vari- 
ance estimates. This underestimation effect will vary from domain to domain 
just as the nonresponse rate varies from domain to domain. The weighting 
class imputation procedure results in a less significant underestimation 
effect for the continuous items than the hot deck procedure, because the impu- 
tation procedure itself contributes less variability than that of the hot deck 
and hence, ignoring the variability added by weighting class imputation has 
less of an effect than when the hot deck procedure is used. 
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Table 4-7 •—Average of the ratios of the STDERR standard deviation 
estimates to the BRR standard deviation estimates for 
the hot deck and weighting class procedures for selected 
domains 



Domain 


Discrete estimates 


Continuous estimates 


Hotkd^ck 


Hot deck 


Weighting class 


Total 


l.*Uz 


0.86 


0.94 


Males. 


1. Oo 


0.80 


0.87 


Individuals of high 

oh i 1 i fv 
au J.11L y 


1* UD 


1.03 


1.05 


Individual * with J.ow 
socio-economic status 


1.05 


0.91 


1.13 


Blacks 


0.97 


0.83 


0.95 


South 


1.07 


1.08 


1.13 


Blacks of average 
ability 


0.96 


0.92 


1.13 



! 

i 



Table 4-8.— The ratio of the STDERR* standard deviation estimat 
to the BRR estimates for TQ141FA 



es 







Hot deck 




Weighting class 


Domain 


STTERR 


BRR 


SD 


STDERR 


BRR 


SD 






SD 


SD 


ratio 


SD 


SD 


Ratio 


Total 




67.93 


84. 55 


0. 80 


63. 61 


70. 11 


0.91 


Sex: 


Male 


79.90 


95.41 


0.84 


76.01 


95.18 


0.80 




Female 


101.89 


172.30 


0.59 


95.47 


138.95 


0.69 


Ability 


: Low 


175. 90 


223. 51 


0.79 


149.76 


150.34 


1.00 




Middle 


104.45 


93. 23 


1:12 


106.37 


77.07 


1. 38 




High 


122.64 


115.58 


1.06 


111.48 


106.99 


1.04 


SES: 


Low 


124. 76 


167. 62 


0. 74 


106.42 


122.43 


0.87 




Middle 


88. 52 


92. 87 


0.95 


82.66 


81.72 


1.01 




High 


106. 48 


153. 47 


0.69 


111.13_110.06' 


1. 01 


Race: 


Black 


157.84 


183.67 


0.86 


147.76 


156.84 


0.94 




White 


73.38 


82.33 


f\ oft 

0. 89 


69.51 


64.46 


1.08 




Hispanic 


853.14 


965.70. 


0.88 


674.42 


980.63 


0.69 




Other 


280.55 


298. 39 


f\ ft / 

0. 94 


268.93 


263.13 


1.02 


Region: 


Northeast 


144.72 


178.95 


"6.81 


148.26 


158.11 


0.94 




South 


133.94 


118. 67 


1. 13 


121.88 


84.12 


1.45 




North central 


124.51 


211, 63 


0.59 


106.64 


145.55 


0. 73 




West 


120.38 


147.77 


0.81 


119.72 


118.44 


1.01 


Race x 


Ability: 














Black: 


Low 


250.55 


247.68 


1.01 


207.14 


140. 71 


1.47 




Middle 


362.80 


417.60 


0.87 


353.30 


382.02 


0.92 


White: 


Low 


207.43 


247. 73 


0.84 


160.00 


145.49 


1.10 




Middle 


112.69 


97.11 


1.16 


115.96 


69.06 


1.68 




High 


124.96 


99.68 


1.25 


113.68 


98.19 


1.16 


Other: 


Low 


633.72 


644.23 


0.98 


624.49 


638.94 


0.98 




Middle 


381.12 


432. 27 


0.88 


356.86 


376.07 


0.95. 




High 


562.22 


569.54 


0.99 


556.75 


515.27 


1.08 



NOTE. — The blacks of high ability domain and the cross classifications 
of Hispanics by ability were omitted from this table since these 
domains had «$uch a small sample size that valid variance estimates 
could not be obtained. 
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V. CONCLUSIONS 

* The results of this methodological study concerning missing and faulty 
•data shculd be interpreted keeping two important facts in mind. First, the 
study was an empirica] investigation based solely on 2p selected critical 
items from the Thi^d Follow-up of the National Longitudinal Survey of the High 
School Class of 1972. The conclusions of this investigation refer to this 
special population of young adults and this type of survey instrument and may 
not be true whta applied to other- populations and to other types of surveys. 
Secondly, it should be emphasized that the study focused exclusively on the 
estimation of univariate means and proportions. If more complex statistical 
N analyses *were to be conducted, the conclusions of this study »T.ight not be 
valid. 

In the experimental data set^which contained the original responses to 
the 20 selected critical items before editing and telephone resolution, in- 
consistent data represented the most serious source of error in the survey 
estimates with the discrete items most susceptible to this kind of bias, es- 
pecially TQ1, TQ9, and those items nested within routing patterns. The effect 
of this source of measurement error bias could be seen in many ways. The 
relative bias of many of the estimates obtained ,using the discrete data was 
over five percent. Due to the large sample size, the variance of the es- 
timates was small so that even a moderate amount of bias became important 
because of its magnitude in relation to that of the variance. This effect was 
measured by the bias ratio for which many estimates had values of 0,5 or more. 
Finally, the best overall measure of the quality of the estimates is the 
relative mean square error, which became quite large for a few of the es- 
timates . 

It is clear that in analysing the NLS data base some attention should be 
given to the problem of resolving logical contradictions contained within an 
individual's responses to the items on the survey instrument. While some 
minor editing has been done with °ags indicating violations of routing pat- 
4 terns, the majority of the editing has been left to the user. From the re- 
sults of this study, one can see that merely discarding the inconsistent re- 
sponses and coding these items as blank is not an acceptable solution. Rat.her, 
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the user of the NLS data base should design edit checks and choose procedures 
for the imputation or correction of missing and faulty data that are best 
suited for the items under study and for the type of analysis being conducted. 

For the discrete items investigated in this study, the hot deck procedure 
did reduce the overall bias of the estimates, but much of the improvement may 
have been related to the editing procedure which removed inconsistent data 
within routing patterns and replaced these responses with data from an indivi- 
dual in the same weighting class who responded in a consistent manner. Except 
for this minor editing, the imputation procedures were not designed to correct 
for what turned out to be the most important source of error, the inconsis- 
tencies within an individual's responses. Procedures have been designed which 
will do both logical editing and imputation of responses for missing and* 
logically erroneous data by Statistics Canada (Hill, 1978). A procedure such 
as their CAN-EDIT might be the best solution for reducing the bias resulting 
from missing or faulty data for discrete items. 

The weighting cla*s procedure devised for discrete items i u this study 
did not work in practice. For categorical items with inissirg responses, the 
weighting class imputation technique randomly assigned responses in such a 
manner that within each weighting class the weighted proportion of nonrespon- 
dents assigned each response option was equal (as far as possible) to the 
proportion of respondents who gave that response (see" Section II. C). In 
practice, the weighting class imputation technique performed poorly since the 
number of nonrespondents to an item within each weighting class was so small 
(often as few as one or two individuals) that it was impossible to have the 
proportion of nonrespondents assigned each response equal \,even approximately) 
the proportion of respondents who gave the response. Retrospectively, one can 
suggest that the distribution of responses should have been estimated for each 
item within weighting classes (as before); then a response should be assigned 
randomly and independently for each missing value in such a manner that the 
probability of each of the ^respdnse options being assigned for the missing 
value would equal the proportion of respondents from the same weighting class 
who circled that response option. Such a procedure mig .t have proved to be 
competitive to the hot deck procedure in reducing the bias. 

For the continuous items, the nonresponse rate was higher but the biasing 
effect of nonresponse was not of significance even for TQ141FA and TQ141FB 
where the nonresponse rates were 13 percent. The effect, of inconsistencies 



in the data set could be seen in the bias of the estimates for the continuous 
items as well, especially for those within routing patterns. The weighting 
class procedure was most effective in reducing the bias of the continuous 
estimates, but the reduction in the relative root mean square error was less 
than 0.5 percent in general. 

At the start of this investigation, the level of nonresponse for the 
selected items before telephone editing was not known. Indeed, the reason why 
so many of the questionnaires failed edit was not known. It might have been 
of general scientific interest to have artificially generated larger nonre- 
sponse rates than those found in this study (for at least a few items). 
However, the results would not have been applicable to the NLS data set. In 
fact, the experimental data set was typical of one type of data set for which 
imputation is commonly used — high rates of response for categorical items, 
inconsistencies in responses associated with routing patterns and items which 
are similar, and higher rates of nonresponse for sensitive items such as 
income. In the past, analysts have behaved as if one should always impute for 
missing responses to "clean up" the data set. This study points out that such 
imputation can reduce rather than increase the accuracy of survey estimates. 

For both the discrete and continuous data, the bias caused by nonresponse 
was insignificant. The bias resulting from individuals failing to interpret 
the questions correctly was large in relation to the nonresponse bias, and 
some attention should be given to this source of error. When using non-crit- 
ical data items which do not undergo such rigorous manual editing, the NLS 
data user should devise logical editing and imputation procedures which fit 
his analysis to reduce such response errors. However, it should be emphasized 
that the data items being studied in this investigation were selected because 
data editors responsible for the manual editing of the questionnaire suggested 
that these items exhibited the largest rates of missing or faulty data. To 
correct for the errors in these critical data items, NLS utilizes the best 
procedure — the individual is telephoned and all missing faulty data is cor- 
rected. \ 
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VI . SUGGESTIONS FOR FURTHER RESEARCH 

There are three topics related to item nonresponse that are of special 
interest and importance to users of the data from the National Longitudinal 
Survey. The topic of more immediate importance is the problem of missing 
ability scores. Approximately 30 percent of the sampled students for whom 
follow-up data was collected have missing ability scores. These students were 
enrolled in schools that refused to participate in the Base Year Survey in 
which the ability scores were obtained. Later many of these schools were 
included in follow-up surveys. Much of the important data that was contained 
in the Base Year instruments was obtained retrospectively during the first 
follow-up, but the ability scores could not be obtained. It is clear that 
some type of special purpose imputation procedure needs to be developed to 
replace the missing ability scores. An empirical investigation would be the 
best way to develop and test imputation procedures. One could create an 
experimental test file by using the data records for students with complete 
ability scores and removing the ability scores for all students within certain 
schools. The pattern of deletion in the experimental data set would be mod- 
eled after the actual pattern which occurs in the NLS data. Various impu- 
tation strategies could then be evaluated in terms of their bias and mean 
square error. Among the techniques that should be investigated would be hot 
deck, weighting class, and regression procedures for replacing missing data. 
The problem posed of imputing missing ability scores for NLS data is difficult 
because of the large rate of missing data, but alternative solutions could be 
evaluated using this empirically based procedure. 

A second topic of interest to users of NLS data would be the effect of 
imputation on multivariate statistics. The present investigation focused 
almost exclusively on univariate means and proportions. However, users of the 
NLS dwta set typically use more sophisticated statistical procedures such as 
regression, factor analysis, and correlation studies, which contain many vari- 
ables. The effect of item nonresponse is cumulative so that even though 
individual items have a large response rate, the number of records with com- 
plete responses to all items entering into the analysis may be so reduced that 
some type of adjustment becomes necessary. Since most of these procedures do 
not have software designed to adjust weights or perform any other adjustments 
needed to correct for the bias due to nonresponse, imputation of missing 
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values may be the only procedure that is feasible. For discrete variables, 
the effect of utilizing hot deck and the weighting class analog of hot deck 
discussed in Section V could be investigated. For continuous variables, 
research needs to be done on possible techniques to use. As discussed in 
Section IV, the hot deck imputations did not improve the estimates of means in 
general and a compensating increase in the variance of the hot deck estimates 
tended to counteract the bias reduction that was occasionally obtained. 
Predicting the effect of hot deck imputations on multivariate statistics is 
difficult, and thus some alternatives to hot deck should be investigated. A 
good procedure to consider would be a regression procedure which estimates the 
missing response based upon other completed data items. The weighting class 
imputation procedure should also be evaluated although it also has deficien- 
cies. In the continuous case, the weighting class procedure imputed the 
estimated respondent mean for the same weighting class for the missing values. 
Thus, every nonrespondent within a weighting class would be imputed the same 
response. This is not a desirable feature of the procedure as far as varia- 
tion and correlation statistics are concerned. However, if the number of 
weighting classes were made large so that the number of nonrespondents within 
each weighting class was small, then this weighting class imputation procedure 
might be a useful tool. One could also overcome the deficiency of the weight- 
ing class imputation procedure in its assignment of means within classes to 
all missing values by imputing the mean plus the product of a randomly gener- 
ated normal standard deviate times the standard deviation of the observations 
within the weighting class. More sophisticated procedures might also be 
explored in which the distribution of responses to the quantitative item was 
estimated for each weighting class and then a response selected at random from 
this distribution to impute for missing values. All of these imputation 
procedures could be evaluated using the experimental data set constructed in 
this investigation. Since the responses for missing items were obtained by 
telephone interview, the bias and mean square error of the estimates could be 
obtained . 

Finally, a third topic of interest to users of NLS data concerns the 
effect of missing data imputation for longitudinal surveys.. In this inves- 
tigation, no previous follow-up or base year data were used to form weighting 
classes or make logical imputations. This procedure of treating the data file 
as if it were cross-sectional is typically used in imputing missing data in 
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longitudinal surveys because no special techniques have been developed for 
such surveys. There is a serious need for further investigation concerning 
how one should use past and future follow-up data for imputation purposes and 
what effect such imputation will have on estimates of stability and change in 
items. An extension of this empirical investigation of imputation procedures 
could be made to account for the longitudinal nature of the data base, A 
two-proaged approach might be made to the problem of longitudinal imputation 
using the experimental data set constructed for this investigation. One could 
use all available data to create logical editing procedures to replace missing 
and inconsistent data in the experimental data set. Another technique needing 
investigation would be the use of previous follow-up data to form weighting 
classes for imputation purposes. Using the data sets developed for this 
investigation, one could determine the effect on the bias and variance of 
estimates when these two procedures are implemented. This investigation 
should include assessing these effects for longitudinal type estimates such as 
change variables. The present data set would allow the estimate of change 
variables for third follow-up versus base year responses for activity states. 
If fourth follow-up data were available, the effect of the imputed third 
follow-up data on later estimates of change and stability could also be evalu- 
ated. 

These three topics are challenging examples of the problems associated 
with item nonresponse and are deserving of consideration. The first would 
furnish a good example of the effects of various imputation procedures when 
used on data with a high level of nonresponse. The second deals with the 
effect of imputation on inferences when more complex statistical analyses ?.re 
performed. The third topic concerns the effect of imputation on the quality 
of change variables estimated using longitudinal survey data. Ail of these 
topics deal with problems which have not been dealt with extensively in the 
statistical literature, and deserve further investigation. 



BIBLIOGRAPHY OF EXTERNAL SOURCES 

\ 

Bailar, John C. and Barbara A. Bailar (1978). Comparison of two 
procedures for imputing missing survey values* Proceedings of the 
American Statistical Asso c iation , Section on Survey Research Methods , 
462-467. 

Bailar, Barbara A., Leroy Bailey, and Carol Corby (1978). A compari- 
son of some adjustment and weighting procedures for survey data. 
Survey Sampling, and Measurement , ed. N. Krishnan Namboodiri, Aca- 
demic Press, New York, 175-198. 

Chapman, David W. (1976). A survey of nonresponse imputation pro- 
cedures. Proceedings of the American Statistical Association , Social 
Statistics Section , 245-251. 

Cochran, William G. (1977). Sampling Techniques . John Wiley and Sons, 
New York. 

Hartigan, J. A. (1975). Clustering Algorithms , John Wiley and Sons, 
New York. 

Hill, Christopher J. (1978). A report on the application of a systematic 
method of automatic edit and imputation to the 1976 Canadian Census. 
Proceedings of the American Statistical Association , Section on Survey 
Research Methods , 474-479. 

Levinsohn, Jay R. , Louise B. Henderson, John A. Riccobono, and R. Paul 
Moore (1978)* .. National Longitudinal Study; Base Year, First, Second , 
and Third Follow-up Data File Users Manual , National Center for Edu- 
cational Statistics, DHEW. 

McCarthy, P. J. (1966). Replication: An Approach to the Analysis of 
Data From Complex Survey s. National Center for Health Statistic^, 
Series 2, No. 14. 



-60- 



Appendix A 
EXECUTIVE SUMMARY 



/ 



Appendix A 

' EXECUTIVE SUMMARY 

la this empirical investigation of alternate item nonresponse adjust- 
ments, two methods, which are frequently used by statisticians to adjust for 
bias induced by item nonresponse were studied. In particular, the hot deck 
and weighting class adjustment techniques vh^re compared using data from the 
National Longitudinal Study of the High School ClasS of 1972 (NLS). Estimates 
obtained using these two techniques were compared with respect to their bias, 
variance, and mean square error to estimates obtained when no item nonresponse 
adjustments were made. 

1. Construction of the Experimental Data Set 

Rather than constructing a data set with artificially induced nonre^ 
sponse, 1 the decision was made to use actual data that contained item non- 
response for which the answers were subsequently obtained by telephone follow 
-up activities. By using data with naturally occurring patterns of item non- 
response, it was felt that a better understanding could be obtained of the 
actual problems associated with item nonresponse and the effect of nonresponse 
adjustments on the precision ot the resulting estimators. Such a data set was 
constructed from the NLS Third Follow-Up (TFU) Survey by taking account of the 
following set of special circumstances. When certain items on the question- 
naire had a missing response or an inconsistent set of responses for one or 
more of these critical items, the questionnaire was marked as having failed 
edit and the individual involved was telephoned and the missing response (s) 
were added. The data records for individuals whose questionnaires failed edit 
contained the responses to these critical items but did not indicate which 
responses were obtained by the telephone interview or what the original re- 
sponses were. 

In order to obtain this information on the responses before telephone 
resolution, the questionnaires were re-examined by data editors and the orig- 
inal responses to the selected critical items recorded. In all, a total of 
10,850 questionnaires failed edit. For reasons of economy, a subsample of 
size 5,854 was selected for re-examination. Twenty key items chosen to be 
representative of the NLS instrument were examined on each of the selected' 
questionnaires and a notation made as to whethe- or not telephoning was neces- 
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sary to obtain a response to that particular item. They include questions 
that have categorical responses including four items which allow the student 
to choose multiple response options. Many should have been answered by all 
survey participants; others applied to subpopulations such as those employed 
or those in school. Some questions were included that came from within rout- 
ing patterns. Other sensitive questions, such as income, which had quantita- 
tive responses were included. 

For each of these 20 items, the status of the response before telephone 
follow-up was determined. A summary of the status of the original responses 
to these items projected to the full sample is given in Table 1. Except for 
two multiple response option questions (TQ1 and TQ9) and four financial ques- 
tions (TQ89HA, TQ89HB, TQ141FA, and TQ141FB), 95 percent of the questionnaires 
contained a response for an item that was consistent with other responses on 
the questionnaire. The highest rates of missing or blank responses were found 
for the income items, TQ141FA and TQ141FB. The multiple response items, TQ1 
and TQ9, had the highest inconsistency rates; that is, TQ1 and TQ9 responses 
were most frequently in conflict with other questionnaire items. The "other" 
category in Table 1 was composed of those who failed an item but could not be 
contacted for telephone resolution. 

The results presented in Table 1 are based upon the subsample of 5,854 
questionnaires, drawn from the 10,850 that failed edit. Adding in the 9,235 
questionnaires which passed edit (and hence had "consistent" answers for all 
of these items) would reduce all of these percentages by about one-half. 
Thus, the data set that was constructed of original responses had a relatively 
sma'i rate of item nonresponse and a somewhat larger rate of inconsistent 
responses. Judging from where the inconsistencies occurred, the major problem 
other than TQ1 and TQ9 appeared to be associated with the routing pattern 
questions. Thus , in reconstructing the data set, the decision was made to 
leave the inconsistent data as observed rather than to code inconsistent items 
as blank. The hot deck problem and the weighting class imputation program 
we.re then written to force consistency on the data by requiring that the 
responses within a routing pattern agree with the lead-in question to the 
routing pattern. In computing the no imputation estimates , no attempt was 
made *o force consistency on the data within records. 



* Table 1.— Classif ication of original responses to the twenty 
j selected critical items 



Item 




Original response 




Consistent 


Blank 


Inconsistent 


Other 


Discrete 










TQl 


92.4 


0.2 


6.7 


0.7 


TQ9 


87.1 


.0.3 


11.7 


1.0 


TQ10 


96.0 


0.9 


2.7 


0.4 


TQl 2 


-97.3 


0.5 


2.0 


0.3 


TQ29 


99.0 


0.5 


0.3 


0.2 


TQ33 


95.1 


0.5 


4.0 


0.4 


TQ51 


97.5 / 


' 0.9 


1.2 


0.4 


TQ52 


96.1 / 


0.9 


•-2.6 


0.4 


TQ66 


94.5 


1.1 


4.0 


0.5 


TQ90 


97.3 


. 1.4 


0.9 


0.4 


TQ101 


98.5 


1.0 


0.2 


0.3 


TQ102 


99.1 


0.2 


0.5 


0.2 


TQ118 


97.1 


2.4 


0.1 


0.4 


TQ129 


99.2 


0.5 


0.1 


0.1 


TQ131 


99.5 


0.2 


0.2 


0.1 


TQ136 


99.6 


0.2 


0.1 


0.1 


Continuous 












70* 0 


0.6 


0.5 




TQl 6 


97.3 


1.8 


0.6 


0.4 


TQ89HA 


92.9 


2.6 


3.8 


0.7 


TQ89HB 


92.5 


2.8 


4.0 


0.8 


TQ141FA 


82.9 


12.$ 


2.4 


1.9 


TQ141FB 


82.5 


12.9 


2.7 


1.9 
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2. The Hot Deck and Weighting Class Imputation Techniques 

The hot deck technique is flexible and relatively inexpensive to run with 
respect to computer time. Before using the hot deck imputation procedure, the 
da^a^ile was sorted into 87 weighting classes ahd then sorted according to 
strata and school within strata. The weighting classes whi,ch were based upon 
the students race, sex, high school grades, high school curriculum, and 
parents' education were originally formed for total questionnaire nonresponse 
adjustments. Th jse weighting ^classej were adapted for item nonresponse impu- 
tation by incorporating certain routing pattern lead-in questions. For each 
weighting class, an initial hot deck was formed by going through the data file 
and recording the first completed response to each item. Then, as the new 
data was processed, the weighting class to which each individual belonged was 
determined. If the item being examined was complete, then that individual's 
response replaced the response stored in the hot deck for that weighting 
class. Thus new responses were supplied for the hot deck as they appeared in 
the data file. When a questionnaire was encountered with a missing item, the 
response in the hot deck for that weighting class was imputed for the missing 
response. Note that since the data file was sorted into weighting classes 
before imputing for missing values, one would expect the hot leek technique to 
obtain much, if not all, of the bias reduction with a somewhat larger variance 
than would have resulted from the weighting class adjustment procedure. A 
more complete description of the hot deck imputation technique may be found in 
Chapman (1976) and Bailar, Bailey, and Corby (1978). 

SThe second item nontesponse adjustment technique used was a "weighting 
f' imputation method. For continuous variables, the weighting class 
imputation technique simply replaced missing values by the estimated respon- 
dent mean for the weight: ng class containing the individual. When a mean or 
total is being estimated, this weighting class imputation technique results in 
the same estimate as that obtained when weight adjustments are made within 
weighting classes. For categorical items with missing responses, the weight- 
ing class imputation technique randomly assigned responses in such a manner 
that within each weighting class the proportion of nonrespondents assigned 
each response was equal (as far as possible) to the proportion of respondents 
who gave that response. Estimates of response level proportions will be the 
same (except for weight accumulations that cannot be expected to break pre- 
cisely at the point desired) as that which would have resulted had a weighting 



3. Analysis of the Data Set 

Estimates of means and proportions were obtained using both imputation 
procedures for the whole population and domains defined by race, sex, ability, 
socioeconomic status, region, and race by ability. 

The variance of the sample means and proportions was estimated by using 
the balanced repeated replication technique (BRR) • BRR utilizes a balanced 
set of half-sample estimates to compute the sampling variance of complex 
statistics. The variability among the replicated estimates approximates the 
desired variance (McCarthy, 1966), In this investigation, 15 equal sized 
super strata were formed jrtft the nonresponse imputation procedures were separ- 
ately applied to the associated set of 16 balanced half samples, insuring that 
the resulting BRR variance estimates reflect the variability induced by the 
imputatioa-^frocedures . 

4. Summary of the Results 

Due to the high item response rates, statistics computed from the pre- 
telephone data set had a relatively small amount of bias wtjen compared with 
estimates using the post-telephone follow-up corrected and completed data. 
The bias that was observed resulted from two response error sources, namely, 
nonresponse and inconsistent responses. In this investigation, no general 
attempt was made to force consistency on the data within a student's record. 
An exception was made in the hot deck and weighting class imputation programs 
which did force the responses to items within a 4 routing pattern to agree with 
the lead-in question to the routing pattern. 

In general, the hot deck procedure did appear to reduce the bias caused 
by nonresponse' for discrete it3ms. The most improvement for bias was seen 
vith respect to Item TQ118 which also exhibited the most nonresponse of the 
di. -ete items. Results for the proportion of students responding "i" to 
TQ are given in Table 2. The table gives the sample size for each domain, 
the crue" value of the statistic estimated using the telephone corrected and 
completed data file, the relative bias of the hot deck (HD) and no imputation 
(NI) procedures, and the root mean square errors of the procedures. Note that 
most, if not all, of the gain in bias reduction from hot decking was lost by a 
corresponding increase in the variance of the estimates. 

The hot deck technique does not appear to perform very well for con- 
tinuous items, including the income questions which had the highest rate of 
item nonresponse. The hot deck imputations did not improve estimates of means 
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Table 2. — A comparison of hot deck Vs. no imputation for TQ110, 
response no. 3 



Subpopulation 


7 

TRUE 


NI 
RB% 


HD 
RB% 


NI 


HD 
/MSE 


Total 




6.42 


1. 16 


0.17 


0.26 


0.25 


Sex: 


Male 


11,44 


0.35 


0.15 


0.46 


0.46 




Female 


1.24 


1.12 


-0.08 


0.15 


0.15 


Race: 


fclack 


9* 72 


3.07 


-1.31 


0.96 


0.96 




White 


5.90 


1.07 


0.52 


0.26 


0.25 




Hispanic 


8.61 


1.53 


12.96 


5.02 


5.12 




ucnei 


6.23 


0.03 


-2.68 


0.73 


0.83 


Ability: 


Low 


7.05 


2.09 


-0.49 


0.59 


0.53 




Middle 


6.27 


0.49 


-0.22 


0.32 


0.35 




High 


5.17 


1.53 


1.67 


0.39 


0.40 


SES: 


Low 


8.33 


0.94 


0.30 


0.61 


0.62 




Middle 


6.46 


1.58 


-0.10 


0.31 


0.30 




High 




J. • JO 






n aa 


Region: 


NE 


5. 36 


1.35 


0.32 


0.92 


0.90 




NC 


5.96 


0.71 


-0.67 


0.47 


0.52 




S 


7 17 

/ . J. / 


n Q7 




n 98 
u • 


U • jU 




W 


7.53 


1.81 


1.00 


0.28 


0.28 


Race x Ability: 












Black: 


Low 


7.58 


5.07 


1.64 


0.95 


0.86 




Middle 


12.21 


-2.57 


-5.4o 


1.81 


2.45 


White: 


Low 


6.67 


1.44 


-0. 75 


0.55 


6.56 




Middle* 


5.90 


0.92 


0.44 


*0.40 


0.41 




High 


5.10 


1.55 


1.70 


0.37 


0.38 


Other: 


Low 


7.49 


0.70 


-2.89 


1.43 


1.52 




Middle 


A. 70 


1.02 


0.00 


1.32 


1.29 




High 


A. 37 


0.70 


0.00 


2.34 


2.32 


NOTE . — The 


domain 


estimates for 


blacks 


of high 


ability and 


all « 



cross- 



small sample si2e prevented the computation of valid variance estimates. 
The estimate obtained using the telephone follow-up corrected and com- 
pleted data (Y_ DTTT? ) is presented with the relative bias expressed as a 
TRUE r 

percentage (RB%) and the root mean square error (/MSE) for the hot deck 
(HD) and no imputation (NI) estimates. 
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item nonresponse. . The hot deck imputations did not improve estimates of means 
in general; and ag*in, a compensating increase in the variance of the hot 
deck estimates tended to counteract the bias reduction that was occasionally 
obtained (see Table 3). 

The weighting class imputation technique performed poorly when applied to 
discrete items* £^rt of the reason that the weighting class imputation tech- 
nique compared poorly in relation to the hot deck and to the no imputation 
techniques for the discrete items was that the number of nonrespondents to an 
item within each weighting class was so small (often as few as one or two 
individuals) that it was . impossible to have the proportion of uonrespondents 
assigned each response equal the proportion of respondents who gave the re- 
sponse. Overall, the weighting class imputation technique performed best for 
the continuous income items, TQ141FA and TQ141FB, which exhibited the most 
nonresponse. The weighting class estimates had somewhat smaller mean .square 
errors than the no imputation and hot deck procedures (see Table 3). / 

Due to the manner in which the data file was constructed, it was rela- 
tively easy to identify inconsistent items. Recognizing that measurement 
errors caused by inconsistent responses constitute an important source of bias 
in the estimates obtained using the pre- telephone file, a new data file was 
constructed which retained all the inconsistent responses from the mail ques- 
tionnaire but had missing items replaced by responses obtained in the tele*- 
phone follow-up. The difference between statistics using this missing-data 
corrected file (referred to as where ME stands for measurement error) aud 
statistics obtained from the fully corrected telephone follow-up data file 
(referred to as Y^^) provides an estimate of the measurement error asso- 
ciated with inconsistent responses. Referring to Table 4 fohich compares these 

two statistics, one can see that the measurement error associated with incon- 

r 

sistent responses had a significant effect for TQl and TQ9 and cross tabula- 
tions involving these two items. TQl and TQ9 were multiple response option 
questions in which the students were instructed to "Circle as many as apply to 
you. 11 The measurement error associated with these results was large and 
positive indicating that' many students failed to circlfe all of the options 
that applied to them. Note that on the far right in Table 4 are the estimates 
NIC* These were obtained using no imputation on the pre- telephoning data set 

4 

where inconsistent responses were recoded as missing da£a. Even for the 
questions iu which inconsistencies were mo*>t common, i.e., TQl and TQ9, Y MTr 



Table 3.— A comparison of hot deck vs. no imputation for TQ141FA 



Subpopulation 


Y TRUE 


NI 
RBZ 


HD 
RBZ 


wc 

RBZ 


NI _ 


HP 

Jmsf 




Total 




70AO 


— P. LL 






7fi 25 


96 54 


70 99 


Ser.: 


Male 


6623 


0,02 


0.70 


0.09 


92.32 


1C6.12 


95.38 




r emale 


7460 


-0.63 


0.61 


-0,39 


146.36 


178.25 


141.99 


Race: 


Black 


5946 


-3.02 


-0,62 


-0.90 


250.67 


187.38 


165.74 




wnitc 


7139 


-0,54 


0.67 


-0*15 


72.54 


95.38 


65.43 




Hispanic 


8927 


-4,01 


-6,08 


-8.71 


1195. 68 


1108.22 


1251.94 




Otfter . 


71 7 A 


Z. • jj 






3^7 94 


353 39 


282.31 


Ability 


: Low 


7993 


0*30 


2.33 


-0.53 


165.32 


291.28 


1S6.29 




iliddle 


7574 


0.74 


0.28 


0.25 


4 100. 94 


95.61 


79.44 




High 


5327 


-0.14 


1.07 


0.37 


111.18 


129.07 


108.87 


SES: 


Low 


7663 


-0.70 


1.18 


-0.64 


136.33 


190.76 


131.91 




Middle 


7585 


-0 47 


0.49 


-0. 35 


89.73 


100.16 


86.08 




High 


5346 


0.52 


0.45 


1.04 


112.84 


155.40 


123.44 


Region: 


NE 


ASA? 


— 1 so 




-O fi2 


189 22 


195 43 

17J«4J 


163.25 


NC 


7353 


-0.79 


0.80 


-0.62 


102.06 


132.73 


95.81 




S 


7109 


0.56 


2.19 


0.59 


161.28 


262.98 


151.61 




W 


7137 


-0.04 


0.34 


0.00 * 


132.00 


149.82 


118.44 


Race x 


Ability: 






• 










Black: 


Low 


6438 


-4.11 


-0.86 


-2.44 


320. 17 


253. 81 


210.98 




Middle 


5646 


3.26 


1.77 


3.61 


446.45 


429.52 


433.24 


'White: 


Low 


8578 


-O.£0 


2.23 


-1.21- 


171.07, 


313.05 


178.87 




Middle 


7687 


0.65 


0.15 


0.16 


91.03. 


97.86 


70.24 




High 


5380 


-0.29 


1.05 


0.17 


101.50 


101.30 


98.62 


Other: 


Low 


7731 


9.84 


7.04 


6.55 


1045.26 


843.39 


815.54 




Middle 


7705 


-0.94 


1.05 


-0.63 


396.58 


439.82 


• 379.21 




High . 


4801 


0.11 


0.72 


-0.10 


520.95 


570.62 


515.29 



NOTE . — The domain estimates for blacks of high ability and all cross-classifica- 
tions of Hispanics by ability are not presented since their small sample size 
prevented the computation of valid variance estimates. The^estimate obtained 
using the telephone follow-up corrected and completed data (Y TRU£ ) is presented 

with the relative bias expressed as a percentage (RB%^ and the root mean square 
error (*/MSE) for the no imputation (NI), hot deck (HD) , and weighting class (WC) 
estimates. 
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Table 4* — A comparison of various estimators of proportions for the 
total population 



Item Response 


TRUE 


Y ME 


Y NI 


Y HD 


Y NIC 


TQ1A 1 


72.29 


70.55 


70.56 


70.55 


72.98 


TQ1C 1 


17.16 


17.09 


17.10 


17.11 


15.62 


TQ1D 1 


4.12' 


4.42 


4.41 


4.41 


3.78 


TQ1G 1 


9.19 


7.94 


7.94 


7.94 


7.96 


TQ9A 1 


67.82 


63.33 


63.29 


63.29 


67.15 


TQ9C 1 


32.15 


31.59 


31.63 


31.63 


29.02 


TQ9D 1 


3.96 


3.97 


3.99 


3.98 


3.23 


TQ9G 1 


7.00 


5.43 


5.42 


5.41 


5.40 


TQ10 1 


61.22 


61.28 


61 28 


61.24 


62.45 


TQ10 2 


13.06 


13.01 


13.03 


13.06 


11.81 


TO 10 3 


1.45 


1.39 


1.37 


1.35 


1.24 


TQ10 4 


24.27 


24.32 


24.33 


24.35 


24.51 


TQ12 1 


25.66 


27.04 


26.95 


26.50 


22.38 


TQ12 2 


7.20 


7.19 


7.24 


7.23 


7.59 


TQ12 3 


67.14 


65.77 


65.81 


66.27 


70.03 


TQ118 1 


92.68 


92.68 


92.59 


92.67 


92.63 


TQ118 • 2 


0.90 


0.91 


0.92 


0.90 


0.88 


TQ118 3 


6.42 


6.41 


6.50 


6.43 


6.49 


TQ129 1 


9.78 


9.80 


9.77 


9.80 


9.73 


TH 19 0 9 




AS QS 


£5 92 


45.92 


45.93 


TQ129 3 


4.02 


4.01 


4.02 


4.01 


4.02 


TQ129 4 


40.25 


40.24 


40.30 


<+0.28 


40.32 


Cross-Tabulativ ns : 




• 








TQlAxTQIC • 


10.21 


9.09 


9.10 


9.11 


9.29 


TQlAxTQID 


2.97 


3.12 


3.11 


3.11 


2.75 


TQ9AxTQ9C 


17.24 


12.99 


12.98 


12.99 


13.68 


TQ9AxTQ9D 


2.68 


2.35 


2.36 


2.35 


2.14 



NOTE. — The estimates given above were obtained using telephone follow-up 
corrected and completed data (Y TR0£ ) , the data with missing responses 

completed but inconsistencies not corrected (Y^) , the pre-telephone 
data when no imputation (Y^) and hot deck (T^j) was used, and the pra- 
telephone data with inconsistencies removed and no imputation procedure 
used (Y NIC i. 
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did not generally produce less biased estimates than (no imputation) and 

Yim ^ot deck) obtained using the pre-telephone data set with the inconsistent 
HD 

cies left in* 
♦5. Conclusions 

In the past, analysts have behaved as if one should always impute for 
missing responses to "clean up" the data set. This study points out that such 
imputation can reduce rather than increase the accuracy of survey estimates. 
For most items, no significant gains in accuracy were achieved by using the 
imputation procedures. In part, this was because the response rates for the 
individual items were quite high. Also, the lack of important gains through 
imputation can be attributed to the fact that a reduction in bias was accom- 
panied by a compensating increase in variance. If the item nouresponse rates 
had been higher and the associated nonresponse bias larger, the effect of bias 
reduction might h%ve more than offset the corresponding increase in the var- 
iance of the statistics. For the continuous items where weighting class 
estimates could be compared with no imputation and hot decking, the weighting 
class estimates did have somewhat smaller mean square errors for the items 
with higher nonresponse rates. Unfortunately, accurate variance approxima- 
tions for imputation-based statistics aire difficult and costly to obtain so 
that most users v?ill ignore the imputation in computing the variance. In a 
sense then, one disadvantage of using imputation techniques will be to un- 
derestimate the true variance of sample statistics, especially when the number 
of values being imputed is large. This underestimation could jeopardize the 
validity of confidence statements (Bailar and Bailar, 1978). 

Finally, it should be emphasized that this study focused exclusively on 
the estimation of univariate means and proportions. If more complex sta- 
tistical analyses were being conducted, such as regression or factor analysis, 
which used many variables, it would be much easier to analyze the data when 
the missing values have been imputed. Also, the effect of item nonresponse is 
likely cumu?.ative so that even though individual items have a large response 
rate, the number of records with complete responses to all the items entering 
into the analysis may be so small that some type of imputation becomes nec- 
essary. The effects of imputation on inference when more complex statistical 
analyses are performed is a topic deserving considerable further investi- 
gation. 
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i Appendix B 

DATA CODING FOR THE ITEM NONRESPONSE IMPUTATION STUDY 

1. Instructions to the Data Editors Re-examining the TFU Questionnaires 

The following items from the NLS Third Follow-up will be examined in this 
investigation: 1, 9, 10, 12, 15, 16, 29, 33, 51, 66, 89a, 89b, 90, 101, 102, 
118, 129, 131, 136, 141a, 141b. In both parts of questions 89 and 141 only 
the total will be examined. 

Only questionnaires fiom the "fail-edit 11 or B batches will be examined. 
To select the mail questionnaires to examine, choose the first questionnaire 
in each batch and every other questionnaire in th^t batch until the ques- 
tionnaires are exhausted. Repeat this process for each of the B batches 
containing mail questionnaires (these have batch numbers less than 500 and a B 
attached to the number). Every questionnaire obtained through a personal 
interview will be examined. 

For each questionnaire selected, record the batch number and student I.D. 
number on one of the specially prepared Key Question Edit Problem Sheets (see 
Attachment B-l). For each of the item numbers circled on the special edit 
problem sheet, examine ^he Key Question Edit Problem Sheet enclosed in the 
questionnaire. If brown ink appears beside one of the circled items on this 
sheet, that question is to be examined. Conditional items within a routing 
pattern will also be checked when the routing item leading to the question is 
flagged. (Sometimes conditional items will be brown inked when the routing 
item is flagged and sometimes not.) 

For each item circled on the special Key Question Edit Problem Sheet, the 
following coding will be used: 

1. Leave the item blank if no telephoning was necessary. 

2. If the item was left blank and an answer supplied by the 
telephone operator, code that item B for "blank." 

3. If the original response was changed by the telephone 
operator, code that item R for "right." 

4. If the original response was changed by the telephone 
operator, code that item by giviag the original response. 

5. If the answer is illegible or if a multiple response 
(when only one was possible) was given, code that item 
as BD for "bad data." 

ERIC " 74 " 



6. If a response of "don't know" was given, code the item DK. 

7. For item number 141, a frequent errof was that respondents 
would give their wages and then leave the others blank, 
including the total income. For this question, if the amount 
given was correct (or if zfcro, blank), but the respondent 
failed to total them, code the item BR for "blank but right 
entries given above." 

8. If the telephone operator was unable to contact the individual, 
code the blank and inconsistent items with UTC for "unable to 
contact." . ^ 

J\ 

2. Original Responses to' the 20 Selected Cruc^J^Ttems 

Following coding by the data editors, freq^Acies were computed for each 
item and each code. These frequencies are pressed as percentages in Attach- 
ment B-2. 



Si 



9 

ERIC 



-75- 



TW flow . 

Kty Question Edit Problem Sheet 



Editor: 
Telephone Operator: 







Pass 


Fail 


BATCH # 










Edit 






I.D.: 


Tracing 






- 


Telephone Edit 







Check all key questions and Important non4cey questions which have caused a problem. When possible, briefly describe each problem. 



65* 
@> 
67 

68 _ 
70 
71 
73 
74 

78* 
83 
84 




91 

92 

93 

94 

95 

96 

97 

98* 

99* 

100* 

® 

© 

q * 103 „. 

0 ERIC 1 

'important non-ktv Questions. 



104 

108 
109 

& 
119 

126* 

121* 

® 
180* 
13J 



182* V 
133* _ 
134* _ 



135* 



m 

<S3>- 

158 . 

FFU25 . 

FFU29a . 

FFU48a . 

FFU54a _ 

FFU64 . 

SFU 1 . 



Section G: Background Information 

Respondent's Name and Address 

Respondent's Parent's Name and Address 

Spouse's Name 



qo 



Attachment B-2, — Item frequencies for the data codes used to describe the original response to 





Che 


twenty 


critical items for the subsample of 


5860 fail 


-edit questionnaires 








Data 


code s for orig ina 1 


response 






Item 


. 

Cons istenC 
(C) 


Blank 
(B) 


Inconsistent 
but correct 
(R) 


Inconsistent 
and incorrect 

(i) 


Bad 

data 

(BD) 


Don t 

know 

(DK) 


Blank but 
right entries 
sriven above 
(BR) 


Unable 

to 
con tact 
(UTC) 


TQl 


86.0 


0.3 


3. 1 


9.2 


0.0 


0,0 


0.0 


l.A 


TQ9 


76.1 


0.5 


A. 6 


16.9 


0.0 


0.0 


0.0 


1.8 


TQ10 


92.6 


1.5 


3.7 


1. 3 


0.1 


u. u 


.0.0 


0.8 


TQ12 


95.0 


0.9 


2. A 


1.2 


0.0 


0.0 


0.0 


0.5 


TQ15 


97.5 


1.0 


0. 5 


0. A 


0.2 


0.0 


0.0 


0.5 


TQ16 


94.9 


3.0 


0. 5 


0.5 


0.2 


0.0 


0.0 


0.7 


TQ29 


98.0 


1.0 


0. A 


0.2 


0.0 


0.0 


0.0 


0.A 


TQ33 


90.9 


0.9 


3.9 


3. 5 


0.0 


0.0 


0.0 


0.8 


TQ51 


95. A 


1.6 


0.9 


1.3 


0.0 


0.0 


0.0 


0.7 


TQ52 


92.9 


1.6 


2.9 


1.8 


I). U 


U. U 


0.0 


-0.8 


TQ66 


89.7 


1.9 


A. 3 


3. 1 


0. 1 


0.0 


0.0 


1.0 


TQ89HA 


86.8 


4.8 


2.8 


A. 2 


. 0.1 


0.0 


0.0 


1. 3 


TQ89HB 


86.0 


5.0 


2.8 


A. 6 


0. 1 


0.1 


0.0 


1.4 


TQVl) 


95.1 


L. A 


0.5 


1.1 


0.1 


0.0 


0.0 


0.7 


TQ101 


97.3 


1.8 


0.A 


0.0 


0.0 


0.0 


0.0 


0.5 


TQ102 


98.3 


0.A 


0. 3 


0.7 


0.0 


0.0 


0.0 


0.3 


TQl 18 


94.7 


A. A 


0. 1 


O.u 


0.0 


0.0 


0.0 


0.8 


TQ129 


98.6 


0.9 


0. 1 


0.1 


0.1 


0.0 


0.0 


0.3 


TQl 31 


99.1 


0.A 


0.2 


0.2 


0.0 


0.0 


0.0 


0.2 


TQl 36 


99.2 


0.A 




fi n 


0.0 


0.0 


0 0 


0.2. 


TQl A I FA 


68.2 


16.5 


1.5 


2.9 


0.2 


0.0 


7.2 


3.5' 


TQl^lFB 


67.6 


17.5 


1.5 


3.5 

( 


0.1 


0.0 


6-3 


3.6 
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Appendix C 
THIRD FOLLOW-UP QUESTIONNAIRE 



0.1 




si* 



Preoartd for ttte 
OEPARTMENT OF HEALTH, EDUCATION, ANO WELFARE 
BY RESEARCH TRIANGLE INSTITUTE Z RESEARCH TRIANGLE PARK, NORTH CAROLINA 

FAIL 1976 "Q^ 

•80- ^° 



W STAST 



SECTION A: GENERAL INFORMATION 



. 1. What wtft you doing tht first wtt k of Octobtr 1f?4? 

• # : 

(Qrclo as many as apply. ) 

Working for pay at a WMime or part-time job I t 

Enrolled in graduate or professional school 2 1 

Taking academic courses at a two* or four-year college 3 

Taking vocational or technical courses at any kind of school j 
or college »for example, vocational, trade, business, or * 
other career training school > 4 

TQ1E On active duty in the Armed Forces < o r service academy » 5 

TQ1F Homemaker $ 

T Q* G Temporary layoff from work, looking for work, or waiting 

to report to work 7 

TQ1H Other (describe: ) ....8 



TQ2 1 How would you dtscrib* your living quarttrs as of thu first wttk of Octobtr 1976? 

(Grclt ont.) 

Private house or mobile home 1 

Private apartment 2 

Dormitory or apartment operated by a school or college 3 

Fraternity or sorority house 4 

Rooming or boarding house 5 

Military service barracks, on board ship, etc 6 

Other (describe: > 7 



TQ3 3. With whom did you livt as of tht first wttk of Octobtr 1976? 

(Grdt on*. } 

By myself 1 

With my parents 2 

With my husband or wife j 

With parents and husband or wife 4 

With other relatives 



With persomsunot related to me 6 



TQ4 4. Which of tht following btst describes tht location of tht plact whtrt you lived in tht first week of Octobtr 1976? 

(Cirelt ont.) 



In a rural or fanning community i 

In a small city or town of fewer than 50.000 people that is not 
a suburb of a larger place 2 

In a medium-sized city < 50.000-100.000 people \ . ? 

In a suburb of a medium-sized city .... 4 

In a iargf city < 100.000-500.000 people) 5 

In a suburb of a large city $ 

In a very large city 'over 500.000 people > 7 

In a suburb of a very large city .... 3 

A military base or station . 3 
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Washington D.C. 20202 



* GENERAL INSTRUCTIONS 

This questionnaire is divided into the following seven sections: 

A. General Information 

B. Work Experience 

C. Education and Training 
0. Military Service 

E. Family Status 

P. Experiences and Opinions 

G. Background Information 

Start by answering questions in Section A. You will need to answer the first question in each section, but you may not 
need to answer all the questions in every section. You may be able to skip most of some sections. We have designed the 
questionnaire with special instructions in red beside responses which allow you to skip one or more questions. Follow 
these instructions when they apply to you. 

Read carefully each question you answer. It is important that you follow the directions for responding, which are: 

• (ClrcJeone.) 

• (Cirde as many as apply.) 

• (Circle one number on each line.) 

Sometimes you are asked to fill in a blank— in these cases, simply write your response on the line provided. 
Where you are asked to circle a number, make a heavy circle. Here is an example: 

Why did you leave high school? (Circle one number on each line.) 

My NOT My 

Reasons Wessons 

Graduated (3 2 

Envered collage 1 (2) 

Went to work • -O 2 

Many questions ask what you were doing during a specific time period; for example. "What were you doing during the 
first week of October 1976?" Because it has been two years smce we last heard from you. we also ask some ques- 
tions about what you were doing in 1975. As you go through the questionnaire, please watch for these time references 
and makesure you are thinking about the correct time period for each question. 



This questionnaire is authorized by law 20 USC I22ie-1 . 

The Federal Pnvacy Act of 1974 requires that each survey respondent be informed of the following. 

(1) Solicitation of information about the respondent as detailed m the questionnaire is authorized by Section 415 of the 
General Education Provisions Act as amended (20 USC 1226b). 

(2) Disclosure ofrthis information by the respondent is subject to no penalty for not providing ait or any part of the re- 
quested information. 

(3) the purpose for which this information a to be used is to provide statistics on a national sample of students as they 
move out of the American high school system into the cntical years of earry adulthood and relate these statistics to 
postsecondary educational costs and financial aid and other factors on the educational, work, and career choices of 
young adults. 

(4) The routine uses of these data will be statistical in nature as detailed in 9 in Appendix 8 of the Departmental Regula- 
tions (45 CRF 56) published in the Federal Register. Vol. 40. No. 196. October 8. 1975. 



When you compiete this questionnaire, please place it in the post-paid, addressed envelope provided and mail it to: 

OPERATION FOLLOW-UP 
Research Triangle Institute 
Post Office Box 12036 

Research Triangle Park. North Carolina 27709 
THANK YOU FOR YOUR COOPERATION 
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J5 5. Is this the SAME city or community where yog livod in October 1f74? 

Yes / 1 GOTOQ.8 

No 2 CONTINUE WITH Q. 6 

\6 *. How far is this from where you lived in Octobor 1974? 

(Crcit ooo. ) 

Less than 30 miles .... 1 

50 to 99 miles '.2 

100 to 199 miles 3 

200 to 499 miles 4 

500 miles or more 5 



A 

CONTINUED 



j 7 7. What was tht main reason you moved to the place where you live now? 

(Circle one.) 



To find or take a job 1 

Was transferred .....2 

Other job-related reason 3 

To go. to school 4 

To follow my parents or spouse to a new location 5 

To follow another relative or friend to a new location 6 

Wanted a better place to live 7 

Other (specify: 1 .. .8 



*. Which of the following items do you have the use of as your own because you (or your spouse) have bought them 
or have been given them, or because they belong to your parents, roommates, dormitory, apartment building, 
etc-? 

(Circle cne number on each Vint.) 



Havt As Hav« But Oon'» Have 
My Own Don't Own Use Of 

8A a. Daily newspaper - I 2 3 

8B b. Dictionary \ 2 3 

8C c - Encyclopedia or other reference books I 2 3 

8D d - Magazines ... 1 2 .. 3 

*8e e. Record player „ \ 0 3 

8F f- Tape recorder or cassette player . . 1 2 3 

8q g. Color televsion 1 .2 3 

8H * h - Typewriter 1 .2 . . 3 

81 1- Electric dishwasher \ .2 3 

8J j. Two or more cars or trucks that run i.. . 2 . . 3 



9. Now please think back a year to Fall 1975. What were you doing in October 1975? 

(Circle as many as apply.) 

9A Working for pay at a full-time or part-time job I 

9B Enrolled in graduate or professional school 2 

9C Taking academic courses at a two- or four-year college .... 3 
Taking vocational or technical courses at any r dnd of school 
or college (for example, vocational, trade, nismess. or 
other career training school ) 4 

9E On active duty in the Armed Forces (or service academy 1 5 

9F Homemaker 6 

9G Temporary layoff from work, looking for work, or waiting 

to report to work 7 

9H Other (describe: > 3 
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SECTION B: WORK EXPERIENCE 

* [ 

* 1 

In this section, wewould like to find out about the jobs you have held in the two-year period from October 1974 through 

October 1976. lnc!ngJull-time jobs, part-time jobs, apprenticeships, on-the-job training, military service and so on. 

We are interested in learning about the kinds of jobs you have held, the hours you worked and your income from these 
jobs, the level of your job satisfaction, and the relation of yoir training and education to vour work experience This in- 
formation will help us better understand the movement of young people into the world of work and '.he reasons for 
changes in job situations. 



JOBS HELD IN OCTOBER 1976 



TQ10 10. ^Oid you hold a job of any kind during the first week of October 1976? 



(Circle on*.) 



GOTOQ 13. next pane 



Yes. working full-time <35 hours or more per week) 

Yes. working part-time (34 hours or fewer per week) 2 

Yes. but on temporary layoff from work or waiting to 

report to work 3- 

510 4 CONTINUE WITH Q. 1 1 



11. What were the reasons you wore not working dumg the first week of October 1976? 

(Circle one number on each line.) 



TQ11A a. Did not want to work 

TQ11B b. Was full-time homemaker 

TQ11C c - Going to school 

TQ11D d- Not^enough job openings available 

TQ11E e. Required work experience I did not have 1 

TQ11F f. Jobs available offered little opportunity for career development 

T(JllG g. Health problems or physical handicap 

TQ11H h. Could not arrange child care 

TQ11I i. Other family responsibilities i including pregnancy* 

TQ11J j Not educationally qualified for types of work available . 

TQ11K k. There were jobs but none where I could use my training . . 

TQ11L I. Spouse preferred that I didn't work 

TQ11M m. Other (specify 



My 


NOT 


Reasons 


My Reasons 


. .1 .. 


2 


1 


2 


...1 . ... 


. .. 2 


. . .1 


9 


...l . .. 


2 


..I ... 


2 


..I ... 


. .. 2 


I . ... 


9 


. I 


. 2 


. .1 .. . 


2 


1 . .. 


. . .2 


. .1 . 


2 


.. 1 


2 



TQ12 ,12- W»r» you looking for work during th« first wek of Octobor 1976? 



Yes 



(Circle ono.) 

..IV 



No. but DID look for work sometime during the month of 
September 1976 

No. and did NOT look for work at any time during the month 
of September 1976 — r 



3 J 



GOTOQ 32. p. 7 
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313A-D 



J13E 



CONTINUEO 



.13, Pteat* describe Mow the job you hold during tho first w»k of October 1974. (If you hold mort than 
ono job at that tiroo, describe tho ono at which you workod tho most hours.) 



a. For whom did you work? (Name of company, business organization* or other employer > 
(Write in): 



b. What kind of business or industry was this? (For example, retail shoe store, restaurant, etc.) 
* < Write in)-- 

c. What kind of job or occupation did you have in this business or industry? (For example, salesperson waitress 
secretary, etc. ) 



(Write in): 



d What were your most frequent activities or duties on this job? (For example, sellinwhoes. waiting on tables 
typing and filing, etc.) ^ 

(Write in): 

e. Were you: 



(Crclo ono.) 

An employee of a PRIVATE company, bank, business, school, or individual working Tor 
wages, salary, ^r commissions? i 

A GOVERNMENT employee (Federal. State, county, or local institution or school)? 2 

Self-employed jn yo^ir OWN business, professional practice, or farm? 3 

Working WITHOUT PAY in family business or farm? 4 



f. When did you start working at this job? 

g. Are you currently working at this job? 



No. 



TQ13FA t month) TQJ.3FB 

2 Date left: TQ13GB (month) TQ13GC 



(year) 



ivear) 



14, How did you find this job? 



14A 
14B 
14C 
14D 
14E 
14F 
14G 
14H 
1'4I 
14J 
14K 
14L 



(Grclt as many as apply.) 



a. 


School or 7 college placement service .... 


1 


b. 


Professional periodicals or organizations 


. . . 2 


c. 


Civil Service applications 




d. 


Public employment sen-ice 


4 


e. 


Private employment agency 


. . . . 5 


f. 


Community action or welfare groups 


. 6 


g- 


Newspaper. TV. or radio ads . . . 


.7 


h. 


Direct application to employers 


. . 8 


L 


Registration with a union ... 


9 


J 


Relatives 


10 


k. 


Friends 


11 


1, 


Other i specify 


». 12 



15 15. How many hours did you usually work at this job 
in an avtragt week? 

Hours per week 



TQ16 
16. 



In an avtragt week, approximately.how much 
did you tarn at this job? (Report your gross 
earnings before deductions. If not paid by the, 
week, pltast estimate.) 



S per week 

< Earnings before deductions > 
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TQ17A 

tI}17B 

TQ17C 
TQ17D 



17. The following are soma genera! things that pooplt do on their, jobs. About how much time did you 
spend on each in tht average work day on your job? 



CONTINUE! 



* Working with things (machinery, apparatus, art 
^ materials, etc.) \ 



(Grclt ono numbtr on etch line.) 
Vary 

Nona Little Som*. 

2 J 



A Grtat 
Oaal 



Doing paperwork (administrative, clerical, computa- 

' tional. etc.) \ 2 3 4 

Working with ideas, thinking t 2 3 ........ . .4 

Dealing with people (as part of the job) 1 2 3 4 



11 a. 



About how many paapla wart tmploytd in tha antirt organization for which you worked? Stata o¥ Federal 
employees giva ttie approximate numbar of paopla in- your. Department, ,a.g., Stato, Commerce, Motor 
Vehicles, ate. Seif-employed giva ma approximate numbar of your amployaas. iCircIa one numbar in 
Column A.) 

About how many of thasa paopla workad in tha 3 am* plant or offica as you? (Circle on* number in Column B.) ' 



A. Total 
Organization 

.. 1 



i worked alone 

Less than 10 2 

10-99 .1 3 

100 - 499 4 

500 - 999 5 

1.000 - 2.499...^ 0 .. 

2.500 and over T\ 7 

o 

TQ18A, 



B. Sam* 
Ptant or Offica 

.... 1 

.... 2 

.... 3 

.... 4 . 

.... 5 

.... 6 

... 7 



TQ18B 



TQ19 W- Waasa think of your supervisor or tha parson who had most control over what you actually did on tha job. Which 
of tha following bast describes how closely this person supervised you? 

N ' • (Circle one.) 

My supervisor decided both what I did and how I did it . . . 1 

* My supervisor decided what I did. but I decided how I did it . . . .2 

My supervisor gave me some freedom in deciding what I did 

and how I did it 3 

......... ^ 

I was more or less my own boss within the general policies 
of the organization 4 

There was no such person 5 



TQ20> 



20. 



How many paopla did you supervise in your job? (Include all persons whose work you supervised as well as theso 
for whose work you were hald responsible.) 1 ' 



people 



er|c 



- 8 9-u; 



21. Hew utisfitd w*rt you with the following aspoct* of this job? 



CONTINUED 



21A 


a. 


21B 


b. 


21C 


c. 


21D 


d. 


21E 


e. 


21F 


f. 


216 


g- 


21H 


h. 


211 


i. 


21J 


j- 


21K 


k. 



(Orel* ont numbtr on tach lint.) 



Vtry 
Satisfitd 



Satisfied 



Oissatisfafc 



Vtry 
Oissatitfitd 



Pay and fringe benefits 1 2 3...: 4 

Importance and challenge 1 t 2 3 4 

Working conditions 1 2 3 4. 

Opportunity for promotion and advancement with this 

employer * I 2 .\.3 t .4 

Opportunity for promotion and advancement in this 

line of work, , t 2 3 ...4 

Opportunity to use pasKraining and education 1 2 .3 . . . . 4 4 

Security and permanence 1 ;. .2 3 4 

Supervisor(s) 1 2 3 ...4 

Opportunity for developing new skills . . . . . .. * . .1 2 3 4 

Job as a whole \ 1 * . . .2 . . .3 4 

The pride and respect I received from my family and 

friends by being in this line of work 1 .2 3 4 



22. Not including on»tht*job or tmptoytr training, did you rtctivt formal instruction to do this kind 0' work? 

No.....: 1 GOTOQ. 27, next page 

♦Yes v 2 CONTINUE WITH Q. 23 



23. Whtrt did you rtctivt this training? 

r (GrcJt as many as apply. ) 

High school I 

Vocational, trade, business, or ^ther career training school 2 

Junior or community college 3 

Four^ycar college or university 4 

Military service : 5 

\ Other (describe: ] » ) 6 



24. What wtrt your txptritnets white working on this job? 



(Grclt ont numbtr on tach lint.) 



, My NOT My 

/ Sxptrifict * Exptntnct 

a. I have been able to apply most of what I learned in-school 1 2 

b. I would have likpd more experience in my training before r I started 
working : k 

t. I received training different from the way it was done on the job 

d. I was trained with tools or equipment not used on my job 

e. I could have gotten my job without the training * 

f. I took coursework associated vyth my tr&ning which was not he pf ui 
in performing my job 

g. Most of what I did on the job I learned to do in school 

h. I consider myself doing as well as others with similar training 

1. I consider going to school and getting the training a wise choice 

1 02 



TQ25 w#ft y° w Wrtd fa* *W* because your tmptoyer 

knew you had been trained in a school or colltgt 
to do this kind of work? 

Yes i 

No 2 • 

Don't know 3 



T026 

21. Did tho school at which you rtcoivod 
your training for this job refer you 
to this job? 



CONTINUE 



Yes 

No.. 



TQ27 27. Do you oxpoct.to bo working in Octobor 1977? 

No 

Don't know 2 , 

- Yes 3 CONTINUE WITH Q. 28 



l 2 }pOTOQ.29 



TQ28 

► 28. Do you f>lan to work at tho SAME KIND OF WORK? 

Yes ^ , 1 

No 2 

Don't know 3 



,TQ29 29. Waro you working at a sacond job in tho first week of Octobor 1976 at tho SAME TIME as you hold tho job you 
doscribod above? 

No..., 1 GOIOQ.32 

Yes ,...2 CONTINUE WITH Q. 30 , — - 



( TQ30 30. How many hours did you usually work at this job 
in an avorago w4ek? 

Hours per week 



TQ31 
31. 



In an avorago woek, approximately how 
much did you earn at this job? (Report your 
gross earnings before deductions. If not paid 
by the week, please estimate. ) 

per week 



i Earnings before deductions i 



JOBS HELD IN OCTOBER 1975 



.a. 



TQ32 32. Now please think back to Fall 1975. Did you hold a job of any kind during the month* of October 1975? 

(Circle one.) 

4 Yes, working full-time (35 hours or more per week) 

Yes. working part-time 1 34 hours to fewer per week) 2 ).QOTOQ 34. next pave 

. Yes. but on temporary layoff from woFk or waiting to 
t - report to work 3 - 

No \ 4 CONTINUE WITH Q. 33 

* TQ33 33. Were you looking fcr work during OctobeM975? 

(Circle one.) 

Yes , 1 

No. but DID look for work sometime during the month of 
September 1975 : 2 

No. and did NOT look for work at any time during the month 
of September 1975 3 J 



> GOTOQ 4L$. 9 



9 
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CONTINUED 



34 34. It thia H»# Him job you httd in October \fH and rtporttd, in Q. 13? * 

1 ■ T^rcleoneJ 

Yes : ^A^GOTOQ.36 

• No, different job 2 

- No. was not working in October 1976 3 



J CONTINUE WITH Q. 35 



35. Picas* dt scribe btlow fy^yob yiu httd duripg October 197S. (If you hold mort than ono job at that time, describe 
tht ont at which you worktd the most hours. ) 

35A-D a - For whom did J™ worlc? <Name of company, business organization, or other employer 1 



35E 



(Write in): 



b. What kind of business or industry wa$ this? (For example, retail shoe store, restaurant, etc. ) 



(Write in); 

.0 



g 1 . ■- 

c. What kind of job or occupation did you have ig this business or industry? (For example, salesperson, waitress, 
secretary, etc. ) 

(Write in): V, 

d. What were your most frequent activities or duties on this job? (For example, selling shoes, waiting on tables, 
typing and filing, etc.) 

i Write in): m . 

ej Were you: 



T 



(Grcie one.) 



An employee of a PRIVATE company, bank, business, school, or individual working for 
wages, salary, or commissions ; I 

A GOVERNMENT employee i Federal. State, county, or local institution or school » 0 2 

Self-employed ui your OWN business., professional practice, or farm? 3 

Working WITHOUT PAY in family business or farm? 4 

<month> TQ35FB (year) 



f. When did you start working at this job? TQ35FA 

±& 

36 36. How many hours did you usually work at this job 
in an average week? 

Hours per week 



TQ37 



37. In an average week, approximately how 
much did you tarn at this job? (Report your 
gross earnings before deductions. If not paid 
by the week, please estimate.) 

i 

$ r ____ per week 

i Earnings before deductions > 



38." Art you currently working at this job? 

Yes 1 GOTOQ. 40^ next page 

No 2 Date left: TQ38B 



TQ38A 



(month) TQ38C 



J year) CONTINUE WITH Q. 39 



ERIC 



-89- 



39. How important wtn tht following as reasons for your Itaving this job? 



CONTINUED 
(Crclt ono numbtr on tach lino.) 



> 






Vtry 
Important 


So mt what 
Important 


Not 
Important 


TQ39A 

, f rt "JOB 


a. 


Poor pay or fringe benefits 








o 
D. 


Lack of importance and challenge 










Sm. 












Q. 


Lack of opportunity for promotion and advancement with this employer 

Lack of opportunity for promotion and advancement with this line of • 
work , 




2 


3 




6. 








TQ39F 


f. 


No or little ODDortunitv to use oast training and pdnr^tinn 


1 


o 




TQ39G 


g. 


Lack of security or permanence 


1 


0 




: TQ39H 


h. 


Dissatisfied with mv ^supervisor ( s > | 


1 




«J 


' TQ39I 


i. 


Lack of ODDortunitv for develoDinff-npw ckilU 


1 


o 




TQ39J 


i. 


UnhaDDv with the 10b as a whole 


1 


o 


3 


TQ39K 


k.' 


Moved to another location ^ 


1 


o 




TQ39L 


!. 


I was laid off or fired. . . . 


1 


o 




TQ39M 


m. 


Went back to school or college . . . 


1 


9 




TQ39N 


n. 


Got married 


1 


9 




TQ390 


0. 


Had a baby 


1 


9 


0 


TQ39P 


p. 


Other family responsibilities ) 


1 


9 


*} 


TQ39Q 




Left to obtain a better job 




2 


3 


TQ39R 


r. 


Health problems or physical handicap 




9' 


.3 


. TQ39S 


s. 


Promotion or transfer within same organization ? 






3 


TQ39T 


t. 


Temporary or school-related job 




2 


3 


TQ39U 


u. 


Others specify: > 


...1 


9 


3 



TQ40 Wort you working at a second job during Hit month of Octobtr 1975 at* tht SAME TIME as the job you 

dtscribtd abovt? - 



41. 



42. 



No.. 
Yes 



During tht two 52-wttk ptriods from (a) Octobtr 1974 to Octobtr 1975 and from(b) Octobtr 197S to Octobtr 1976 
how many different employtrs did you work for altogtthtr? ( Count each tmpfoytr only onct, tvtn if you had dif. 
ftrtnt jobs for tht stmt tmploytr.) 



(a) 

Octobtr 1974- 
Octobtr 197S 

TQ41A 



Number of employers 



(b) 

Octobtr 1975- 
Octobtr 1976 

TQ41B 



Number of employers 



Durinsrtht stmt two 52-wttk ptriods from (a) Octobtr 1974 to Octobtr 1975 and from (b) Octobtr 4975 [o Octobtr 
1974, about how many wttks did you wor* altogtthtr? (Count alt wttks in which you did any work at all or wtrt 
on paid vacation.) 



(a) 

Octobtr 1974- 
Octobtr 1975 

TQ42A 



weeks 



(b) 

Octobtr 1975- 
Octobtr 1976 

TQ423 



weeks 
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CONTINUED 

In each of these 52-week periods from (a) October W4 to October WS and from (b) October 197S to 
Oct obor 1*74, wort thort any wtoks in which you wort NOT working and wtro looking for work, 
on layoff from a job, or waiting to roport to work? 



i4X 44. What kind of work wilt you bo doing whtn you art 30 years old? (Crclo the one that eomos elosost to what you 
i4Y expect to bo dning. ) 

(Cede one.) 

i a. CLERICAL such as bank teller, bookkeeper, secretary, typist, mail earner, ticket agent i 

b. CRAFTSMAN such as baker, automobile mechanic, machinist, painter, plumber, telephone 

installer, carpenter 2 

c. FARMER. FARM MANAGER \ : : 3 

d. HOMEMAKER OR HOUSEWIFE ONLY 4 

e. LABORER such as construction worker, car washer, saniiary worker, farm laborer 5 

f. MANAGER. ADMINISTRATOR such as sales manager, office manager, school administrator. 

buyer, restaurant manager, government official 6 

g. MILITARY such as career officer, enlisted man or woman in the Armed Forces 7 

h. OPERATIVE such as meat cutter, assembler, machine operator, welder, taxicab. bus. or truck 

driver, gas station attendant 8 ■ 

i. PROFESSIONAL such as accountant, artist, registered nurse, engineer, librarian, writer, social 

worker, actor, actress, athlete, politician, but not including public school teacher . 9 

j. PROFESSIONAL such as clergyman, dentist, physician. lawyer, scientist, college teacher 10 

k. PROPRIETOR OR OWNER such as owner of a smali'business. contractor, restaurant owner . . II 

I. PROTECTIVE SERVICE such as detective, police officer or guard, sheriff, fire fighter 12 

m. SALES such as salesperson, advertising or insurance agent, real estate broker 13 

n. SCHOOL TEACHER such as elementary or secondary 14 

o. SERVICE such as barber, beautician, practical nurse, private household worker, janitor, waiter . 15 

p. TECHNICAL such as draftsman, medical or dental technician, computer programmer 16 

q. NOT WORKING ' IT 

\ 

+5 45. Do you think you will need mora education or schooling than what you have at present in order to obtain this kind 
of work or to advance as you would like in your job or career? x 



Yes 2 

Don't know 3 

How satisfied 'are you with the progress you have made towards doing tto 
when you are 30 years old? 

(Circle one.) 

Very satisfied I 

Satisfied 2 

Dissatisfied 3 

Very dissatisfied 4 

. for- coding instructions. 
# 



SECTION C: EDUCATION AND TRAINING 



START 



This section asks information about your training and education. We would like to find out about the schools you have 
attended during the last two years, from October 1974 to October 1976. This information, combined with information you 
have given us in earlier follow-ups. will help to give us a complete picture of your educational experiences since high 
school, i Persons in-the military service should also answer the questions in this section. > 



EDUCATIONAL PROGRESS AND PLANS 



TQ47 47. Sinct high school, had you tarned any ctrtificate, license, diploma or dogrtt of any kind prior to October 1976? 

No I GOTOQ.49 . 

Yes 2 CONnNVEWUHQ.48 

4a. What kind of etrtifieatt, license, diploma or degrtt have you tarned? 

Art* of Cartiftcatt, Lictnst. 
IGrelt Data or Dtgrtt (For ExjmpU, Rt*I 

as many Rtttived Estatt Lictnst, Shorthand 

as apply.) Month Ytar Ctrtif:catt, Otgrtt in History) 

TQ48*A A certificate 1 TQ48AB 19 TQ4 8AC TQ48AD ' 

TQ48BA . A license ' -.2 TQ48BB » TQ48BC TQ48BD 

* TQ48CA A 2-year or 3-year vocational . " " 

degree or diploma 3 T048CB 19 TQ 48CC TQ48CD 

TQ48DA A 2-year academic^egree 4 *' TQ48DB 19 TQ 48DC TQ48DD 

TQ48EA A 4-year or 5-year college 

Bachelor s degree 5 TQ48EB 19 TQ 48EC TQ48ED 

TQ48TA . A Master's degree or equivalent * 6 - TQ48Fg 19 TQ 48FC TQ48FD 

TQ48GA Other ispecify: i ..7 TQ48GB 19 TQ48GC TQ48GD 



4?. a. As of tht first week of October 197t, what was your highest Itvtl of education or training? (Column A) 
b. As things stand now, how far in school do you think you actually will got? (Column B) 

TQ49A TQ49B 



A. 
Had in 

t ^ Octobor \m 

(Circltont.) 

Finished high school j 

Vocational trade or 

business school .. .( Less than two years 2 ... 

1 Two years or more 3 . . . 

Less than two years of college 4 

Two or more years of college < including 
two-year degree) 5 ... 

FThished college ifour- or 
five-year degree) 5 

Master's degree or equivalent 7 

Ph.D.. or advanced professional degree . . / . 8 . 



College program 



a. / 

P lan to gtt> 
(6 re It ont.) 

' 1 * 

... 2 
... 3 
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SO. With re§ard to your education and training during the last year you wort in school, 
Sow satisfied a* a whole wort you with tho following? 



51. 



(Grde ono numbor on each lino.) 



Very 
Satisfied 



Somowtiot 
Satisfied 



Noutrol 
or No 

Opinion 



Sjmewhot 
D'ssetisfitd 



SOA 


a. 


SOB 


D. 




C. 


SOD 


d. 


SOE 


e. 


50F 


f. 


50G „ 


g- 


50H 


h. 


301 


i. 


50J 


j- 


10K 


k. 



The ability, knowledge, and personal 

qualities of most teachers l 2 3 4 . 

The social life 1 .! 2 3 4. 

Development of my work skills 1 2 3 4. 

My intellectual growth 1 2 3^ 4. 

Counseling or job placement 1 v 2 3 4 . 

The buildings, library, equipment, etc 1 2 3 4 

Cultural activities, music, art. drama, etc 1 2 3 4 . 

The intellectual life of the school 1 2 3 .... / 4 

Course curriculum 1 2 3 4 . 

The quality of instruction 1 2 3 \4 . 

Sports and recreation facilities i ■ .2 3 4 . 



c 

CONTINUED 



Very 
Dissatisfied 



5 
. .5 
. .5 
. 5 
. ,5 
. .5 
. .5 
. .5 
. .5 
. 5 

:5 



During tho rwo-yoar poriod from October 1974 through October 1976 wore you enrolled in or did you take clashes 
at any school like a college or university, graduate or professional school, service academy or school, business 
school, trade school, technical institute, vocational school, community college, and so forth? * 



No 1 GOTOQ. 98. p. 22 

Yes 2 CONTINUE WITH Q. 52 



SCHOOL ATTENDANCE IN OCTOBER 1976 



52. Oid you attend school in tho first week of October 1976? 

No 1 GOTOQ. 66. p. 15 

Yes 2 CONTINUE WITH Q. 53 



53. What is tho exact name and location of the school you were attending in tho fi rst week of C^ber 1976? (Please 
print and do not abbreviate. ) 



School Name: 

City: TQ53B 



TQ53A 



State: T053C 



54. What kind of school is this? 

' (Grcleone.) 

Vocational, trade, business, or other career training school 1 

Junior or community college i two-yean 2 

College or university < four years or more) 3 

Independent graduate or professional school < medical. 

dental, law. theology, etc. I 4 

Other (describe: i ...5 



1U3 



9 

ERIC 
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J 



51 Whtn did yog first atttnd this school? TQ55A (month ) TQ55B t year ) 



CONTINUE) 



So. Art you currently attending this school? 

Yes l 



TQ56A 



No. 



.2 Date left: TQ56B 



'month) TQ56C 



i>ean 



TQ57 



During tho first wook of Octobor W6, war* you classified by this sdtool as a full-timo studtnt? 

Yes it. 

Dontknow 2 \OOTOQ.59 

No 3 CONTINUE WITH Q. 58 



58. 



TQ58A 


a. 


TQ58B 


b. 


TQ58C 


c. 


TQ58D 


d. 


TQ58E 


e. 


TQ58F 


f. 


TQ58G 


g- 


TQ58H 


h. 


TQ58I 


t. 


TQ58J 


j- 



What wtrt ypur reasons for attending school part-ttma instead of full-timo? 

(Crclo as many as apply.) 

Could not afford to go full-time 1 

Working f&lUime 2 

Working part-time 3 

Family responsibilities 4 

Taking job-related courses 5 

Taking courses for personal enrichment 6 

Undecided about career plans 7 

Too much pressure or strain with full-time load 8 

Health problems or physical handicap 9 

Other (specify: * >....10 



TQ59 



59. 



During October 1976, about how many hours a wotk did your classes moot in the subjects or courses in which you 
were enrolled? include time in lectures, shop, laboratories, etc* 

hours pet* week 



TQ60 60. At that time how were you classified by your school? 

>' (Circle one.) 

Freshman Ifirst-year undergraduate student) 1 

Sophomore (second-year undergraduate student » 2 

Junior (third-year undergraduate student) 3 

Senior (fourth-year undergraduate student) 4 

jfe Graduate or professional student 5 

Special student 6 

Other classification (specify: > ....7 

My school doesn't classify students 8 



•94- 



CONTINUED 

41. As of tM first week of October 1*74, what was your actual or intended field of study or 
training ar*a (for example, practical nurse, machinist beautician, civil engineering, 
accounting* psychology, homo economics, etc.)? 

Please name tho specific fit Id or arta : 

(Write in): I 



41 Pleese soloct btlow Hit category which best describes this field or area. 

(Circle one.) 

Agriculture or Home Economics 1 

Business (accounting, marketing, personnel management, etc. > 2 

Office and Clerical (bookkeeping, stenography, general office, etc. > 3 

Computer Technology (keypunch operator, programming, computer operations, etc.* 4 

Education « elementary, special, physical, etc. ) : 5 

Engineering (civil, electrical, mechanical, etc.) 6 

Mechanical and Engineering Technology (automotive mechanic, machinist, construction, drafting. 

electronics, etc.) 7 

Humanities and Fine Arts t music, religion. English, etc.) 8 

Health Services (nursing, lab technician, occupational therapy, etc.) 9 

Public Services daw enforcement, food service, recreation, beautician, etc. i 10 

Physical Sciences and Mathematics ( physics, geology, chemistry, etc. > U 

Social Sciences (psychology, history, economics, soc clogy. eiv.. > 12 

Biological Sciences (zoology, physiology, anatomy, etc. ) 13 

Professional Program (medicine, dentistry, law. theology, etc. i H 

OTHER field or area t specify : TQ62FS » ■■ 15 

UNOECIOEO - 16 



63. This (above) is; 



(Grcfe one.) 



A vocational program . 1 

An academic program -2 

A professional program 3 

Other (specify: » ■ ■ 4 



64. As of the first week of October 1976, what kind of certificate, license, diploma, or degree were you studying for* 

(Grcfe as many as apply.) 



}64A None 1 

J64BA A certificate (specify in what: TQ64BB >. . . 2 

}64CA A license (specify in what : TQ64CB > .... 3 

}64D A 2-vear or 3-year vocational degree or diploma 4 

}64E A-2-year academic degree 5 

}64F A 4-year or 5-year college Bachelor's degree 6 

J64G A Master's degree or equivalent 7 

J64H A Ph.D. or equivalent 3 

>64IA An M.D.. L.L.B.. B.D.. D D.. D D.S.. or equivalent 

(specify degree: TQ64TK 1 •■• 9 

J64 JA Other i specify : TQ64JB » • • • ■ W 



er|c 



tin 
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i 

i TQ6S Curing Octobtr 1974, did ytw work for tht school you wtrt attending? 

(Grcleont.) 

Yes. working for pay 1 

Yes. working off cost of tuition, housing, or meals 2 

Yes. both of the above 3 

No ' • 4 



SCHOOL ATTENDANCE IN OCTOBER 1975 



TQ66 $4. Now pltasa think back to Fall 1975. Wtrt you taking classts or courses at any school during tht month of 
October 1975? * 

No 1 GOTOQ. 79.j>. 17 

Yes. at the same school I attended in October 1976 and 

reported above in Q. 53 2 GOTOQ. 70 

Yes. at a school I have not yet reported 3 CONTINUE WITH Q. 67 

67. What is tht exact namt and location of tht school you wtrt attending in Octobtr 1975? (Pltast print and 
do not abbreviate. ) 

' .» 

School Name: TQ67A 



City: T067B State: TQ67C 

TQ68 6*. What kind of school is this? 

4 (Circle one.) 

Vocational, trade, business or other career training school 1 

Junior or community college i two-year » 2 

College or university (four years or more* 3 

Independent graduate or professional school i medical. 

dental, law. theology, etc. > 4 

Other i describe: > ..5 

*f. When did you first attend this school? TQ69A < month > TQ69B 'year » 

TQ70 70. During Octobtr 1975, wore you classified by this school as a full-time student? 

Yes 1 

No 2 

Don't know 3 

v/ . 

TQ71 71. During Octobtr 1975, about how many hours a week did your classts meet in tht subjects or courses in which you 
wtrt enrolled? include time in lectures, shop, laboratories, etc. 

hours per week 
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CONTINUED 

At that time how were you class ifiod by your school? 

(Grele one.) 

Freshman ( first-year undergraduate student ) 1 

Sophomore ( second-year undergraduate student) i -2 

Junior t third-year undergraduate student ) 3 

Senior tfourth-yjear undergraduate student) % . 4 

Graduate or professional student - . 5 

Special student 6 \ 

Other classification (specify: J ••■• 7 

My school doesn't classify students 8 

o 

Was your Hold of study or training arta in Octobtr WS the samt as it was in October 1976? 

(Grcleone.) 

yes \ m ~ ' 1 GO TO Q. 77, next page 

■ No, was not in school in October 1976, 2 1 CONTINUE WITH Q. 74 

No. different from October 1976 3 J 

* As of October 1975, what was your actual or intondod field of study or training area (for example, practical 
nurse, machinist be autician, civil engineering, accounting, psychology, homo economics, etc.)? Please name the 
specific field or arta: 

tWritein): , 



i 

75. Ploaso soloct bolow the category which bett describes this field or area. 

(Grcleone.) 

Agriculture and Home Economics 1 

Business (accounting, marketing, personnel management, etc.) 2 

Office and Clerical t bookkeeping, stenography, general office, etc. ) 3 

Computer Technology (keypunch operator, programming, computer operations, etc.) 4 

Education (elementary, special, physical, etc.) •'• • 0 

Engineering (civil, electrical, mechanical, etc.) 6 

Mechanical and Engineering Technology (automotive mechanic, machinist, construction, drafting, 

electronics, etc. ) 

Humanities and Rna Arts i music, religion, English, etc.) 8 

Health Sorvkes (nursing, lab technician, occupational therapy, etc. ) 9 

Public Sorvicas (law enforcement, food service, recreation, beautician, etc. > 10 

Physical Sciences and Mathematics (physics, geology, chemistry, etc.) U 

Social Sciences (psychology, history, economics, sociology, etc. ) 12 

8ioiogical Sciences < zoology, physiology, anatomy, etc. ) ...... 13 

Professional Program (medicine, dentistry, law, theology, etc. ) • • • 14 

OTHER field or area (specify: T075?? ] " Ao 

UNDECIDED ; " 16 



)76 74. This (above) is: 

(Grcleone.) 

A vocational program 1 

An academic program ■ • • 2 

A professional program 3 

Other (specify: ! ) ... 4 
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77. As tf Octobtr 1f7S # what kind of ctrtificate, lictnst, diploma, or dogma wort you studying far? 

(Grclt as many as apply.} 

TQ77A None , 1 

TQ77BA 'A certificate (specify in what : j TQ77BB ) .... 2 

TQ7 7CA A license (specify in what : TQ77CB ) .... 3 

TQ77D. A 2*year or 3-year vocational degree dr diploma 4 

TQ77E A 2*year academic degree 5 

TQ77F A 4-year or 5*year college Bachelor s degree 6 

TQ77G A Master's degree or equivalent 7 

TQ77H A Ph.D. or equivalent 8 

TQ77IA An M.D., L.L.B.! B.D.. D.D.. D.D.S.. or e^ivalent ) 

(specify degree: TQ77TB >. • - . 9 

TQ77JA Other (specify: TQ77JB >.. . . W 

TQ78 71. During October 1975, did you work for the school you wtrt attending? 

H (Grclt ont.) 

Yes. working for pay : 1 

Yes. working off cost of tuition, housing, or meals 2 

Yes. both of the above 3 

No „ 4 



CONTINUE! 



SINCE OCTOBER 1974 



TQ79 79. Has your field of study or training area changtd at any firm since Octobtr 1974, two years ago? 

No 1 GOTOQ.81 

Yes ....2 CONTINUE WITH Q. 80 



Listtd below aro somo rtasons why students change fields or training antes. What wtrt tht rtasons 
in your situation? 

(Grclt ona numbtr on tach lint.) 



TQ80A 


a. 


TQ80B 


b. 


TQ80C 


c. 


TQ80D 


d. 


TQ80E 


e. 


TQ80F 


f. 


TQ8pG 


g- 


TQ80H 


h. 


TQ80I 


i. 


TQ80J 


j- 


TQ80K 


k. 



Courses more difficult than I expected 

Met people with new ideas 

Poor advice on original choice 

Lack of information on jobs related to original choice 

Content of courses different from what I expected 

New information abcut other fields of study or training areas . 

Interest aroused by courses 

More jobs available for graduates in the field I changed to . . . 
Better jobs available for graduates in the field I changed to . . 

Interest aroused by job I have held 

Other (specify: 



My 
Reasons 



NOT My 
Reasons 

... .2 
2 

.... .2 

.J .2 

2 

2 

2 

. .. .2 
2 
.2 

2 



TQ81 H. Have you changtd schools at any timt sinct Octobtr 1*74, two ytars ago? 

No 1 GO TO Q. 83. next page 

Yes 2 CONTINUE WITH Q. 82 
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CONTINUED 



•2. What were your reasons for changing schools? 

(Crete ont numbor on each lino.) 

My NOT My 

Rwom Reason* 

Enrolled in graduate or professional study at another school I 2 

My interest changed, and my former school did not offer the course of ^ 

study I wanted , 1 " 

Wanted to attend a less expensive school I 2 

Wanted to be at a smaller school | • • 1 2 

Wanted to be at a larger school 1 2 

Wanted to attend school closer to home 1 - 

Wanted to attend a school farther away' from home I 2 

Wanted to atjtend a school that would give me better career opportunities . . : 2 

Wanted to attend a more prestigious school 1 2 

Wanted to attend a school where I could maximize my intellectual 
and personal development 



12A 

9£A 


a. 




b. 


82C 


c. 


82D 


d. 


VZE 


c. 


82F 


f. 


82G 


g 


82H - 


h. 


821 


i. 


82J 


j- 


82K 


k. 


82L ', 


1. 


82M 


m. 


82N 


n. 


820 


0. 



i 2 

9 



Transferred from a two-year to a four-year school to continue my 

education • 1 

1 2 



More group or social activities of interest 1 

Transferred from a two 

education 

Family responsibilities 

Health problems or physical handicap - 

Other (specify: , ••• 2 

[83 83. Sinct Ccfobor 1974, have you withdrawn from any school btfort you computed your studios at that school? 
t (Circle one.) 

No i GO TO Q. 85. next page 

Yes. but I have since returned to school . . .' 2 1 

Yes. but I plan to return before October 1977 3 ( CONTINUE WITH Q. 84 

Yes. and I do not plan to return before October 1977 4 J 



What were your reasons for withdrawing? 

(Circle one number on each line.) 

My NOT My 

Retsons Reasons 



a. Health problems or physical handicap I 2 

b. Had financial difficulties # . 1 2 J 

c. Was offered a good job 1 * 

d. Got married or planned to get married i 2 

e. School work was not relevant to the real world i 2 

f. Wanted to get practical experience 1 2 

g. Failing or not doing as well as I wanted i 2 

h. Wasn't really sure what I wanted to do i 2 

i. Transferred to another school i 2 

j. Family responsibilities 1 

) ....I 2 



9 



k. Other (describe: 
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CONTINL 

EsHmaH how wtll you have done in all of your coursework or programs since high school and until 
October 1976. Do not include gradts from graduate or professional school. (Grclt ono numbor in Column A.) 

f.ViT^ tf h PL wtl .! yo SU? v f *"* your «•«"»•«•■* or programs only in tho lyear period from Octobor 
1974 through Octobor 1976. Do, not includt grados from graduate or professional school. (Circlt ono numbor 

mw0,umnBJ " TQ85A TQ85B 

*■ v 
From High Octobor 197^ 

School to J Octobor 1976 
Octobor 1976 

Mostly A (3.754.00 grade point average) \ \ 

About half A and half B (3.25*3.74 grade point average) 2 2 I 

Mostly B (2.75-3.24 grade point average) 3 3 

About half B and half C (2.25*2.74 grade point average) 4 4 

Mostly C (1.75-2.24 grade point average) 5 ' 5 

About half Cf and half D i 1.25-1.74 grade point average ) 6 6 

Mostly Dor below (less than 1.25) 7 7 

Have not taken any courses for tfhich grades were give 8 ' 



TQ86 



Considering all of tho schools you have attended 
since high school, do ANY of those schools or pro*, 
grams give credits which can bo used for a 4-year 
college Bacholor's degree? 



3 



GOTOQ.88 



I don't know 

No 

Yes 3 'CONTINUE WITH Q. 87 



► o7. Since leaving high school, about how many 
credits had you earned by Octobor 1976? 
(Write in J 

TQ87A Number of quarter hours 

TQ87B Number of semester hours 

TQ87C Number of other type of credits 

~ (specify type: » ) 



SCHOOL FINANCES FROM FAIL 1974 THROUGH SUMMER 1976 





S ERLC 



The following questions ask about your school finances for the two time periods of <a> Fall 1974 through Summer 1975 
and ib> Fall 1975 through Summer 1976. Please make sure you answer each questiori for both time periods. If you are 
unsure about the actual dollar amount for a particular item, give your best estimate. 

tt. Wert you in sc&tf at any time during either of tho twelve-month periods from (a) Fall 1974 through Summer 
1975 or (b) Fall 1975 through Summer 1976? " 

<•> / ' (b) 

F all 1974 * Summer 1975 Fall 1975 * Summer 1976 

TQ88AA YCS 1 How many months? TQ88AB Yes 1 How many months? TQ88BB 

No 2 TQ88BA No 2 
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If Considering the two time peripds of (a) Pall 1974 through Summer 
Summer Wo, what is your estimate of how much it cost for you to 
o f who jfjjf l Estimate tho amounts 'and record thorn Wow. Entor 
Do not include costs aftor Summor 1974- Rtcord your expenses for 



Tuition and fees * 

Books and supplies 

Transportation to-arid from class from 
where I lived while attending school 

Other school-related expense's 

Housing and meals while enrolled in school . 
All other expenses while enrolled in school : 

medical, dental expenses, debt payments. 
. insurance, taxes, child care, etc 

. HOW MUCH MONEY IS THIS IN TOTAL? 



CONTINUED 

\fj$ and (b) Pali 1975 through 
live and go to school, regardless 
a tore, "0," where you had no oxpenses. 
tho time you wore in school only. 

(a) <»> 
Pall 1M4 • **« WW • 

Sumwar WS Summor 

$ T089AA S T089AB 

S, TQ89BA S TQ89BB 

$ TQ89CA 1 $ TQ89CB \ 

S TQ89DA S TQ89DB \ 

S TQ89EA s TQ89EB 

$ TQ89FA ...S TQ89FB * 
. TQ89GAA.GAB TQ89GBA.GBB 
$ TqaowA § T083HB — 



SCHOLARSHIPS, FELLOWSHIPS, GRANTS, AND BENEFITS- 



90 W. 



91. 



Considoring th. two tim. priori, of (a) Pa.. 1974 throuoh Summor 1975 and (b) Fall 1975 through Summor 1976, 
did you rocoivo any kind of scbolarimp, fellowship, grant, or. benefits to go to schoof ? 

No 1 GOTOQ.92 

Yes, Fall 1974 • Summer 1975 ■ 2 } ' 

■ Yes.Fall 1975 - Summer 1976 -r 3 CONTINUE WITH Q. 91 

Yes, both of these periods 4 

Estimate tho amounts for each scholarship, fellowship, grant, or benefit you received, and record them below. 
Enter a zero, "0," where you received no financial assistance. Do not include loans. 

(a) <b> j 



\ 



a. 
b. 
c. 
d. 
e. 
f. 

g- 

h. 
i. 
j- 



Basic Educational Opportunity Grant : 

Supplemental E.ducatioftal Opportunity Grant 

College scholarship or grant from college funds 

ROTC scholarship or stipend 

Nursing Scholarship Program 

Social Security Benefits ifor students 18-22 who are 
children of disabled or deceased parents) 

Veterans Administration War Orphans or Survivors 

Benefits Program 

Veterans Administration Direct Bertefits (GI Bill) . 

State scholarship 

Other scholarship or grant iwrite in: 



Pell 1974 . 
Summer 1975 



Fall 1975 • 
Summer 1976 



S TQ91AA S TQ91AB 

S *Q91BA S TQ91BB 

S TQ91CA S TQ91CB 

S TQ91DA 5 TQ91D3 

$. TQ91EA S TQ91EB 



S T091FA S TQ91FB 

$ TQ91GA $ T091GB 

S T091HA $ TQ91HB 

$ TQ91IA S TQ91IB 

$ TQ91JA S TQ91JB 



TOTAL DOLLAR VALUE 



TQ91KA s TQ91KB 



LOANS 



Q92 92. Considering the same two periods from (a) Pali 1974 throuoh Summer 197S and (b) Fall 197S through Summer 
1976, did you receive a loan to go to school? 

1 GO TO Q- 94, next page 

Yes. Fall 1974 • Summer 1975 2 } 

Yes. Fall 1975 - Summer 1976 ... 3'} CONTINUE WITH Q. 93 

Yes. both of these periods 4 

ERIC -loi- 

autaaje manual for coding instructions. 



n. 



TQ94 



CONTINUE 



Estimate the amounts for tach loan you rtctivtd and record thtm below. Enttr a zero, "0," where 
you received no loan. 



(a) 
Fall Iff*. 
Sum mer 1975 



(b) 
Fill 1*75* 
Summtr 1976 



Federal Guaranteed Student Loan 

State loan 

Regular bank loan 

National Defense (Dirtct) Student Loan 

Nursing Student Loan 

School or college loan 

Relatives or friends 

Otherloan (write in: 



T093AA $ TQ93AB 

T093BA S T093B3 

TQ93CA « TQ93f!B 

TQQ3DA '....$ TQ93DB 



TQ93EA $ TQ93EB 

TQ93FA ...$ TQ93FB ; 

TQ93GA $ TQ93GB 



TOTAL DOLLAR VALUE 



TQ93HA $ TQ93HB 

TQ93IA $ TQ93IB 



FINANCIAL ASSISTANCE FROM RELATIVES OR FRIENDS 



□ 



94. Considering Hit two time ptnods of (a) Fall 1974 through Sum mar 1975 and (b) Fall 1975 through Summer 1976, 
did you receive financial assistance (not a loan) from any relatives or friends to go to school? 

No ;.l COTQQ. 96 

Yes, Kail 1974- Summer 1S75 2^ 

Yes, Fall 1975 - Summer 1976 3 1 CONTINUE WITH Q. 95 

Yes, both pf these periods 4 J 

95. Estimate tho amounts you rtctivtd and record them below. Enter a xero, "0," where you received no financial 
assistance. 



(a) 
Fall 1974. 
Summtr 197$ 



(b) 
Fall 1975* 
Summtr 1976 



Parents 

Husband or wife 

Other family or friends . 



TOTAL DOLLAR VALUE 



i TQ95AA 5 TQ95AB 

$ T095BA $ TQ95BB 

$ TQ95CA $ TQ95CB 

$ TQ95DA $ TQ95DB 



MONEY YOU HAD SAVED OR EARNED | 



TQ96 9*. Considering Hit same two periods from (a) Fall 1974 through Summer 197 5 and (b) Fall 1975 through Summer 
1974, did you pay any of the costs to go to school from money you had savtd or tarntd? 

No > 1 (JO TO Q. 98. next page 

Yes, Fall 1974 - Su nnw 1 975 2] 

Yes, Fall 1975 - Summer 1976 ! f . .3 \ CONTINUE WITH Q. 97 

Yes, both of these periods 4 J 



97. Estimate the amounts and record below. Enter a zero "0," where you received no money. 



Own savings or summer earnings 

College work-study or cooperative education program 

Teaching or research assistantship 

Other earnings while taking courses 



TOTAL DOLLAR VALUE 



(a) 
Fall 1974- 
yjmmtr 1975 



(b) 
Fall 1975* 
Summtr Iff 6 



$ TQ97AA $ TQ97AB 

$ T097BA $ TQ97BB 

$ TQ97CA $ TQ97CB 



S TQ97DA $ TQ97DB 

$ TQ97EA $ TQ97ES 
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CONTINUE^ 



SCHOOL FINANCES PROM PALL 19ft THROUGH SUMMER 1977 



98 ft. Art you or will you bo in school at any timo from Pall 1f74 through Sumfflor 1977? 

No 1 )gotoq.ioi 

Don't know 2 J 

Yes % 3 CONTINUE WITH Q. 99 

F 

99. What is your estimate of how much it will cost for you to live and 90 to school this year, regardless of who pays? 
Estimate your expenses and rtcord thorn bo low. Enttr a zero, "0," wtitre you oxooct no trpansas. 



99A tuition and fees $ 

99B Books and supplies $ 

99C Transportation to and from class from 

where I live while attending school $ 

99D « Other school-related expenses $ 

99E Housing and meals while enrolled in school .... $ 

99F •» All other expenses while enrolled in school : 
medical, dental expenses, debt payments, 
* insurance, taxes, child care <?tc $ 

99GA,GB 

99H .HOW MUCH MONEY IS THIS IN TOTAL? .... $ 



100. How aro you mooting (or planning to moot) those expenses? Estimate the amounts you txpoct to recti ve from 
each source and rtcord them below. Enter a ztro, "0," where you expect no money. 



* Amount will rtcotvt 
from tach sou rco 

100A • Grant $ 

lOOBf Fellowship , $ 

100C Scholarship $ 

100D Loan ?. $ 

100E Teaching or research assistantship $ 

J.00F Job other than assistantship $ 

100G Spouse's income S 

;100H Savings ' t, % 

•1001 Parents $ 

: 100J Other relatives or friends $ 

llOOIC Other (specify: ).... $ 



GRADUATE OR PROFESSIONAL SCHOOL 



1 0)1 



Have you received a Bachelor's degree from a four-yoar 0011090 of university? 

No 1 GOTOQ. 108, p. 24 

Yes 2 CONTINUE WITH Q. 102 



)102 192. Did you formally apply for admission (fill out a form and send it in) to any graduatt or professional school 
at any timo before October 1976? 

No 1 GOTOQ. 104, next page 

Yes 2 CONTIhUEWITHQ.103 



>ee manual for coding instructions. 



1M mt -t CONTINU 
103, PIMM H$t btlow tho graduate institutions to which you applied, tht city arid state in which 

tfit institutions art located, and tho dopartmtnt or program (e.g., law school! public health, 

journalism, psychology). < 

At tho right circlt tho number for y% or no if you (a) w*re accopttd; (b) applkd for financial assignee 
such as a grant, fellowship, loan, toaching or research assistantship, etc.; (c) were offered financial I assistance* 
and (d) tnrollod. 



First Choice 

School: 

City: 



T01Q3ASC 



TQ103ACT 



Department or program: 
Second Choice . 



School: 
City: 



TQ103BSC 



TQ103BCT 



Department or program: 
Third Choice 



School: 
City: 



TQ103CSC 



(a) 
Was 
Accepted 



(b) 
Applied 

for 
Financial 
Assistance 



(c) 
Was 
Offered 
Financial 
Assi stance 



_ TQ103AA TQ103AB TQ103AC 

State, TQ103AST Yes....l Yes....l Yes....l 

TQ103AFS No 



No. 



No. 



(d) 
Enrolled 



TQ103AD 

Yes....l 
No 2 



TQ103BA TQ103BB TQ103BC TQ103BD 



TQ103BFS 



-State: TQ103BST Yes....l 
No ....2 



Yes....l 
No 2 



Department or program: TQ103CFS 



TQ103CA TQ103CB 
State: TQ1Q3CST Yes....l Yes....l 



No.. 



No. 



Yes....l 
No 2 

TQ103CC 
Yes....l 
No 2 



Yes....l 
No.... 2 

TQ103CD 

Yes....l 
No ... . 2 



TQ104 104. Circle the category that describes your present status with respect to graduate or professional school. 

(Circle one.) 

I have attended graduate or professional school but 



am not presently attending 



t 4t ^ . , / GO TO Q. 106. next pave 

I am presently attending graduate or professional school 2 J 

I have never attended graduate or professional school 3 CONTINUE WITH Q. (OS 



TQ105A 
TQ105B 

TQ105C 
TQ105D 

TQ105E 

TQ105F 

TQ105G 
TQ105H 

TQ105I 
TQ1C5J 
TQ105K 
TQ105L 



105. Which of the following factors aro important reasons for your not attending ^r^\ff9 or professional school? 

(After you have answered this question, go to Q. 108. next page.) 

(Crete as many as apply.) 

a. I have no interests graduate or professional education .... i 

b. I have family responsibilities that require my presence 

at home 2 

c. I was refused a loan 3 

d. I cannot financially afford to attend graduate school at 

the present time 4 

e. I can earn a satisfactory income without attending 

graduate school 5 

f . I want additional work experience before applying to 

graduate school 6 

g. My career goals are very uncertain 7 

h. I do not need an advanced degree to succeed in the field 

I am now in (or want :o enter* , 8 

i. I'm tired of school 9 

j. I was not accepted at the institution of my choice 10 

k. Health problems or physical handicap n 

I. Other (specify: V K...12 



> COTOQ. 108 next page 



'er|c 



CONTINUED 



104. How important wa» oach t< mo following roaoons in your 

deciding to attond graduatt or profouional ichool? ^ (Crcto ono numoor on .ach lino.) 



Ootormming ™ ™J 

p»cfc Important Important Cotwdor 

1 2 3 4 

06A Obtaining credentials for a specific career 1 ■' : • 3 ; ^ 

,06B No satisfactory jobs available 1 •• 3 " ". 4 

.06C Better salary 1 3 ........... .4 

.06D Enjoyschool . : 3 7. 4 

.06E . Interest in subject matter 3 " A 

,06F Better job opportunities Jl 3 4 

.06G Other ispecify: : _ 1 " l '/" 



107. 



How important wa. oach of the folding roason. in your choosing th. institution in which y^ ar. (w.ro) 
•nrollod for your graduat. or profossional study? ^ ^ ^ |jn# , 



L07A 


a. 


L07B 


b. 


L07p 


c. 


107b 


d. 


107E 


e. 


L07F 


f. 


107G 


g- 


L07H 


h. 


L07I 


i. 


107J 


j- 



Cost of attending 

Availability of financial aid 

Recommendation of undergraduate professor 

Presence of a particular professor at the institution 

Quality of a particular department 

Reputation of the institution 

Location 

Library facilities 

Proximity to spouse's school/ work 

Other ispecify. . 



Oif^mining Not OidNOT 

««ctor Important Important Con»idor 

1 2 3 4 

1 2 3 4 



.1 



.3 4 



.1 2 3 4 

.1 2 3 4 

.1 2 3 4 

.1 2 3 4 

.1 2 3 4 

.1 2 3 4 

2 3 4 



.1 . 
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j OTHER TRAINING 

c« <v*.h.r ma havo vou participatod in any program such as on-tho-iob training, rogistorod appronticoships, 
gp^IS anric^onTor corr^pondonc. coursos? Oo not indue r^u.ar school 

and colltgt programs. 

No 1 GO TO Q. 116. next page 

Yes 2 CONTINUE WITH Q.M 

* 

109, What type of training program (s) or courst(*> havt you partfcipattd in? 

(dixit as many as apply.) 

An Armed Forces training program 1 

On-the-job training (a program of instruction during 

normal worldng hours) 

Employer-provided program of instruction other than - 

on-the-job training J 

Formal Registered Apprenticeship (your state or labor ^ 

union) 

Manpower Development and Training (MDTA) 5 

Wot* Incentive (WIN) 6 

Neighborhood Youth Corps (NYC) 7 

h. Comprehensive Employment and Training Act (CETA) .... 8 

Other manpower program (specify , 1 • • 9 

Correspondence course(s) 10 

Non-credit courses for personal enrichment > - » 

Other (specify: ] ' A1 

-105- 120 



t 


— - 


>109A 


a. 


1109B 


b. 


1109C 


c. 


J109D 


d. 


1109E 


e. 


}109F 


f. 


J109G 


g- 


U09H 


h. 


H09I 


i. 


}109J 


j- 


H09L 


1c 


H09L 


1. 


9 




ERJ.C 





TQ110 "1 Wtrt you boing trainod for mtm typo of work? 

No i GOTOQ.I12 

Yes 2 CONTINUE WITH Q. Ill 



CONTINUE 



TQ111 111. What type of work woro you boing trainod for or Itarning about? If you havo participa'od in mora than ono pro- 
gram, ms\Mr for tho ono in which you spont tho most timo. ( Examples ; plumbing, typing, auto mochanic work, 
photography)^h, otcJ 

; Write in): ^ 



TQ112 112. How long is (or was) this program schodulod to 
last? 

(Cir:to ono.) 

Less than one mon^ i ■ 

One to five months 1 

Six to eleven months 3 

One year or more 4 



TQ113 

113. Havo you complotod this program? 

(Circlt ono.) 

Yes l 

No, left without completing ... .2 
No. still enrolled 3 



TQ114 114. Havo you ustd this training on any job? 

.Yes ,.1 

No .'.2 



TQ113 115. Which ono of tho following statomonts host doscribos tho assistance you rocoivod (aro rocoiving) from tho pro- 
gram or training cantor ?r finding a job? 

( (Crcloono.) 

DOES NOT APPLY TO ME since my training was in the military, on-the- 
job, or for persctwi enrichment 1 

I did not want or aid not need help from the center in finding a job 2 

I wared and needed help but did not receive any from the center 3 

The center provided information on job openings in my field 4 

The center put me directly in touch with possible employers or arranged 

a job for me 5 



TQ116 1U. Havo you ovor triod to find work on a job whoro you hiight uso what you Itarnod from any school, collogo, or 
training program you aftondod sinco Octobor 1*74? 

No. because I have NOT attended any school or college since October 1974 .... 1 1 

No. although I HAVE attended a school or college since October 1974 2 ) SKIP TO SECTION D. next page 

Yes 3 CONTINUE W1THQ. 117 

TQ117 117. Did you find work for which you could uso what you loarnod? 

(Circlt ono.) 

Yes. in the locality u -»re I received my training 1 



\ 



Yes. somewhere else 2 

Yes. both of the above 3 

No : 4 
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SECTION D: MILITARY SERVICE 



118 lit Sine* 0<tobf~1T74, have you »rved in tho Armtd Forces, or a Reserve or National Guard Unit? 

(Grclt fne.) 

„ No ; I [} SKIP TO SECTION £. next page 

Yes. National Guard or Reserves but not active duty 2 J 

Yes. active duty? 3 CONVNOE WITH Q.119' 



119 119. In *hich branch of the Armed Forces drd you serve? (Write in): ^_ 

m Whan did you begin active duty? TQ120A (month) TQ120B (year) 

121 121. Hava you received (or are you receiving) four or more weeks of specralixed schooling while in the Armed 

Fortes? 

No ,...1 GOTOQ.I23 

' Yes 2 CONTINUE WITH Q. 122 

122 122... What is the name of the specialized schooling program in which you spont the longest poHod of time? (Please 

print and do nnt abbreviate.) 



Name of program: 



TQ124 



124. What is the highost pay grado you have 
heM? 

Pay grade : 



123 123. • Specify your current primary military specialty 
code (Army-MOS, Air Force-AFSC,Marines-MO$, 
Navy-NEC). (Please print and use standard ab- 
breviations.) „ , , . . 

Specialty Code: 

125. Have* you taken any courses while in the Armed Forces that: 

(Circle one number on each tine.) 
Yes No 

125A Prepared you for the high school equivalency test? 4 . 1 2 

125B Prepared you for equivalency tests that can be taken for college credit? . . 1 2 

125C Were college-sponsored courses which gave college credits? 1 2 

126. Are your currently on active duty? 

N*m Date left: TQ126B month ' TQ126C year) i SKIP TO SECTION E. next p??e 

TQ126A Y < 2 CONTINUE WITH Q. i27 

127 127. How long do you expect to be on active duty in the Armed Forces? 

(Circle one.) 

For a two-year tour of duty only 1 

For a three- or four-year tour of duty 2 

For nuJre than one enlistment, but less than a full career 3 

For a full career i20 years minimum) 4 

Have not'decided 5 

121. What do you plan to do when you get out of the Armed Forces? 

(Circle one number on each lint.) 
6 My NOT My 

Plant Plans 

128A Full-time or part-time work 1 2 

>128lT Graduate or professional school, either full-time or part-tine 1 2 

>128C College, eifher full-time or part-time 1 2 

;128D Technical, vocational, or business or career training school, either 

fulHime or part-time ; 1 2 

»128E Registered apprenticeship or on-the-job training program 1 2 

il28F • Retire 1 2 



SECTION E: FAMILY STATUS 



START 



TQ129 W. What was your marital status, as of the first wok cf Ocfobtr W4? 



Never married, but plan to be married within the next 
12 months 



(Crde one.) 



1 

Never married, and don't plan to be married within the next 

12 months 2 

Divorced, widowed, separated 3 

Married ; 4 



GOTOQ. 137. next page 
CONTINUE WITH Q. 130 



130. What was tho dato of your marriage? 

T013QA 



i month ) TQ130B 1 year > 



131. 
TQ131BOX 

TQ131A 
TQ131B 
TQ131C 
TQ131D 



TQ131E 
TQ131F 
TQ131G 

TQ131H 



A * of *ho first wotk of October 1»76, what was your husband or wife doing? 

(If you were not married in the first week of October 1976, check here Q and go to Q. 136, next page.) 

(Grcle as many as apply.) 

Working for pay at a full-time or part-time job 1 

Enrolled in graduate or professional school 2 

Taking academic courses at a two- or four-year college 3 

Taking vocational or technical courses at any kind of school 
or college (for example, vocational, trade, business, or 
other career training school ) 4 

On active duty in the Armed Forces ior service academy) 5 

Homemaker 5 

Temporary layoff from work, looking for wort or waiting 

to report to work 7 

Other (describe: j ,,,,3^ 



132. 
TQ132BOX 

TQ132A 
TQ132B 

TQ132C 
TQ132D 
TQ132E 



Please describe btlow tho job your husband or wife helo during tho first week of October 1976. 
(If your spouse was not working, check here Q and go to Q. 135. next page.) 

a. For whom did he/she work? (Name of company, business organization, or other employer) 
(Write in): 

b. What kind of business or industry was this? (For example, retail shoe store, restaurant, etc.) 
(Write in): 



c. What kind of job or occupation did he, she nave in this business or industry? (For example, salesperson 
waitress, secretary, etc.) 

(Write in): 

d. What were his/her most frequent activities or duties on this job? (For example, selling shoes waiting 
on tables, typing and filing, etc: ) 

(Write in): 

e. Was he/she; 



An employee of a PRIVATE company, banc, business, school, or individual working for 
wages, salary, or commissions? 

A GOVERNMENT employee ( Federal. State, county, or local institution or school > 0 . 

Self-employed in his/her OWN business, professional practice, or farm? 

Working WITHOUT PAY in family business or farm? 



(Grclo one.) 

.1 
0 



Q133 133* How many hears did ht/sho usually work at this job in an average week? 

Hours per week 

Q134 TJ4 - ta an ^veraga weak, approximately how much did he/she earn at this job? (Report his/her gross earnings before 
deductions. If not paid by the week, please estimate.) 

S per week 

(Earnings befoi*. deductions) 

Q135 1& As^f Octobor 1f7s, what was tho highest level of education Hut your husband or wife had ^imd? 

(Grele one.) 



Some high school, or less 1 

Finished high school 2 

Vocational trade or 

. . . f Less than two years 3 

business school I * j 

ITwo years or more 4 J 

Less than two years of college 5 

Two or more years of college (including 

~ „ two-year degree) 6 

College program J _ ^ 

S Finished college ( four- or 

five-year degree) ....7 

Master s degree or equivalent 8 

Ph.D.. or advanced professional degree 9 



E 

CONTINUED 



Now please think back a year to Fall W5. What was your husband or wife doing in October 1975? 
(If you were not married in October 1975. check here Q and continue with Q. 137.) 

(Grcie as many as apply.) ^ 

Working for pay at a full-time or part-time job I 

Enrolled in graduate or professional school 2 

Taking academic courses at a two- or four-year college 3 

Taking vocational or technical courses at any kind of school 
or college (for example, vocational, trade, business, or 

other career training school) 4 

On active duty in the Armed Forces (or service academy) 5 

Homemaker .^V- & 

Temporary layoff from work, looking for work, or waiting 

to report to work 7 

Q136H Other (describe: ) ....8 

Q137 137. Aro you a twin? 

Yes I 

No 2 



ERIC 
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TQ138A How many children alt09ether do you eventually axpect to have? 



TQ138B 



.1. 



(Circle one.) 

•3 4 5 6 7 8 or more 



b. As of tho first wook of Octobor 1976, how many childrtn did you have? 

(Grcloono.) 

0 1- 2 3 4 5 6 or more 



CONTINU 



TQ138C c Whon do you expect to havo your first (next) child? 

(Circlo one.) 

Don't expect to have a (another) child r 

Within the next year 2 

Between one and two years from now 3 

Between two and three years from now 4 

Between three and five years from now 5 

More than five years from now 6 

Don't know 7 



TQ139 139. Not including yourself, how many parsons wort dependant upon you for mora than one-half of their financial 
support as of tho first wook of Octobor 1976? 

(Crclo one.) 

0 1 2 3 4 5 6or more 



140. As of tho first wook of Octobor 1976, wore you dependant upon your parents, spouse, or any othor relatives or 
friends for more than on*- ha If of your financial support? 

(Circlo one numbtr on oach lino.) $ 
Yas No 

TQ140A Parents 1 2 

TQ140B Spouse 1 2 

TQ140C Other relatives or friends 1 2 

141. What is tho bast estimate of your income beforo taxes for (a) ALL OP 1975 and for (b) ALL OF 1976? If m*rr\*d, 
include your spouse's income in tho total. Do not include loans. Please make a dollar amount antry on aach lino. 



If you did not rocaivo any income from a source, enter a rare, "0. 



Your own wages, salaries, commissions, or net income 
from a business or farm 

Your spouse's (husband or wife) wages, salaries, com- 
missions* or net income from a business or farm . . . 

Public assistance or welfare (include spouse's) 

Unemployment compensation (include spouse's) 

All other income you and your spouse received (include 
interest, dividends, rental property income, gifts, 
scholarships, fellowships, etc.) 



(a) 

Amount Racaivtd 
1975 



(b) 

Amount Will 
Racaiva 1974 



$ TQ141AA TQ141AR 

$ TQ141BA $ TQ141BB 

$ TQ141CA $ TQ141CB 

$ TQ141DA ..$ TQ141DB 



$ TQ141EA .$ TQ141EB 



TOTAL INCOME FOR YOU AND YOUR SPOUSE 



$ T0141FA $ TQ141FB 



-no- 123 



142. Aooftho first week of October Wfc how much 
money did you owt for: 



(Crcloono number on each line.) 



L42A 
L42B 
L42C 



Other debts (car. rent, appli- 



CONTINUED J 





Less 


$100 


$500 


$1000 


$2000 


$5000 


$10,000 




than 


to 


to 


to 


to 


to 


or 


None 


#1 Aft 

$100 




JAM 


5177? 


54777 


57T77 


more 








....3... 


4... 


...,5... 


....6... 


7 












....5... 


....6... 








9 




4... 


....5... 


....6... 


. ...7 



143. As of the first week of Octobor 1976. how much 
money had you saved and pi an nod to us* for: 



L43A 
L43B 



(Circlo ono numbor on tach lino.) 



Education or training 

Other plans lor general savings) 



lass 


S100 


$500 


51000 


52000 


than 


to 


to 


to 


or 


Horn 5100 


S49f 


$m 


sim 


Mori 


...0 


....2.... 


....3... 


....4.... 




...0 1.... 


....2... 


... 3... 


....4... 





L44 144. Do you owt any monoy for an education or training loan for which your repaymont schedule has begun? 

No 1 SKIP TO SECTION F. next page 

Yes 2 CONTINUE WITH Q. 145 



145. Whtn was your first paymont duo? 

TQ145A (month) TQ145B iyean 



\ 

146 146. Are you having or havo you had any difficulty in mooting payments? 

No 1 

Yes 2 (explain why: 



126 



9 

ERIC 
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SECTION F: EXPERIENCES AND OPINIONS 


1 

START 


. t 

i 


147. To what extent have you voluntarily participated in tht foHowing groups during tht two-year period from Octobor v 
W4 through Octobor W4? <Bv voluntarily, wo mm»n yoiPir* mt » .mpU Y ~ «t ^ y^-^ |ffjvt pirticj 
pant, wo moan that you attond tho meetings or events; by mombor only, wo moan that you aro on a mailing or 
telephone list so that you aro kept informod of mootings and ovonts.) 




i 




(Grclo one number on each lino.) 

Active Mtmbtr Not 
Participant * OnJv At All 


TQ147A 


a. Youth organizations— such as Little League coach, scouting etc 








TQ147B 


b. Union, farm, trade or professional association 










TQ147C 


c. Political clubs or organizations 




1 






TQ147D 


d. Church or church-related activities (not counting worship services ) • 


I 






• TQ147E 


e. Community centers, neighborhood improvement, or social-action 
associations or groups 








TQ147F 


f. Organized volunteer work— such as in a hospital 




! 






TQ147G 


g. A social, hobby, garden, or card playing group 










TQ147H 


h. Sport teams or sport clubs 










TQ147I 


i. A literary, art, discussion, music, or study grouo 










TQ147J 


j. Educational organizations— such as PTA or an academic erouD . 








TQ147K 


k. Service organizations— such as Rotary. Junior Chamber of 
Commerce. Veterans, etc 










TQ147L 


1. A student government, newspaper, journal, or annual staff 










TQ147M 


m. Another voluntary group in which I participate 












143. How do you fool about each of tho following state, nents? 






i ' 








(Grclo one number on each lino.) 




TQ148A 


Agroi 

Strong 

I take a positive attitude toward myself l 


Ajrtt 


OUagrto 


Disagree 

Strongly 


No 
Opinion 


TQ148B 


Good luck is more important than hard 










TQ148C 


I feel I am a person of worth, on an equal 

plane with others 1 










TQ148D 


I am able to do things as well as most 

other people 1 










TQ148E 


Every time I try to get ahead, something 

or somebodv stops me 1 . 










TQ148F 
— TQ148G- 


Planning only makes a person unhappy 

since plans hardly ever work out anyway l 










People_who_acceptJheir_conditioninJife 

are happier than those who try to 










TQ148H 


On the whole. I'm satisfied with myself l 








. . .5 


! o 
ERJ.C 
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CONTINUED 



149. H*ve you ever tetn given a special advantage or treated unfairly because of your sax (mala 
or femala) in any of tha following situations? 



gjvan SfiocUj Advanteqa 
(Grcle m many esepply.) 



Getting a good education i IQ149AA 1 

Getting a job. promotion, or other work benefits 2 TQ149BA 2 

Getting a house or apartment 3 3 

Nor^of these. X , 4 XQ1.49.PA 4 

If so. please describe : TQ149E 



Trsattd Unfairly 
(Grcle as many as apply.) 

TQ149AB 
TQ149BB 
TQ149CB 
TQ149DB 



150. How do you fael about each of tha following statements? 

(Grcle one number on each line.) 



Agree Disagree 
Strongly Agree Disagree Strongly 

150A J a. A working mother of pre*school children can be just as 

good a mother as the woman who doesn't work 1 2 3 4 

150B b. It is usually better for everyone involved if the man is 
the achiever outside the hon$ and the woman takes 

care of the hqme and family \ 1 2 3 4 

150C c. Young men should be encouraged to take jobs that 
are usually filled by women (nursing, secretarial 

work, etc.) I 2 3 4 

150D d. Most women are just not interested in having big and 

important jobs % . 1 2 • . .3 4 

150E e. Many qualified women can't get good "jobs : men with the 

same skills have much less trouble I . . . 2 3 4 

150F f- Most women are happiest when they are making a home 

and caring for children 1 1 2 3 4 . 

150G 8- High school counselors should urge young women to 

train for jobs which are now held mainly by men 1 . . . . ; 2 3 o 4 

15 OH h. It is more important for a wife to help her husband than 

to have a career herself 1 2 3 4 

1501 u Schools teach women to want the !*ss important jobs 1 2 3 4 

150J J- Men should be given first chance at most jobs because 
they have the primary responsibility for providing for 

a family 1 ,■ 2 3 4 



151. How important is each of tha following to you in your life? ^ 

(Grcle one number on each line.) 



Very Somewhat Not 

« t %m Imflortanf Important Important 

'15 1A Being successful in my line of work I 2 3 

1151B b. Finding the right person to marry and having a happy family life 1 2 3 

|15 1C c. Having lots of money 1 2 3 

J151D d. Having strong friendships • ■ • 1 2 3 

{15 IE e. Being able to find steady work 1 2 3 

>151F f. Being a leader in the community 1 2 3 

>151G g. Being able to give my children better opfKJrtunities than I've had 1 2 3 

)151H h. Living close to parents and relatives 1 2 3 

)151I i. Getting away from this area of the country 1 2 3 

)151J j. Working to correct social and economic inequalities 1 2 3 

)151K k. Having leisure time to enjoy my own interests 1 2 3 

}151L 1. Having a good education : 1 2 3 



151 u^J^X*^ °* ,h# ,0,lowiB « *■«*•■* determining the kind of work you plan to bo doing 
for mo* of yowr lift? ^ 

(Oreloono numbor on each lino.) 



CONTINUl 



t 

t 

I 

T0155A 


a. Previous work experience in the area 


Very 
Impertent 


Somewhat 
Important 


Not 
Important 


] TQ152B 


b. Reiativi or friend in the same line of work . 










c. Job openings available in the occupation 








TO! 5?T* 


d. Work nv tches a hobby interest of mine .... 










f e. Good income to start or within a few years 










f. Job security and permanence 








TQ152G 


g. Work that seems important and interesting to me 








TQ152H 


h. Freedom to make my own decisions . . . 








TQ152I 


i. Opportunity for promotion and advancement in the long run 








TQ152J 


j. Meeting and working with sociable, friendly people 










,$3 ' ^^[!° 1 J!f 9 ^ stlon$ " k W P«M**i participation. Considering tht period from October 1974 to 


i 

TQ153A 


When you talked with your friends, did you ever talk about public 
problems—that is. what's happening in the country or in your 
community? ^ 


(Circle jne number on each line.) 
Frequently Sometimes Never 

) 

1 o * 


TQ153B1 


Did you ever talk about public problems with any of the following people? 
Your family 








TQ153B2 


People where you work 








-TQ153B3 


Community leaders, such as club or church leaders 










Did you ever talk about public problems with elected government officials 
or people in politics, such as Democratic or Republican leaders 0 








T0153D 


Did you ever talk to people to try to get them to vote for or against 
a candidate? 








TQ153E 


Did you ever give any money or buy ticket* to help someone who was 
trying to win an election? 








TQ153F 


Did you ev«r go to any political meetings, rallies, barbecues, fish fries, or 
things like thai in connection with an election? 








TQ153G 


Did you ever do any m ork to help a candidate in his campaign? 








TQ153H 


Did you ever hotf an office in a political party or get elected to a 
government job? . 






i 



TQ154 154. 



Aro you registered to voto? 

Yes l 

No 2 



TQ155 155. Bofare Octobor We, did you ovor veto in a ttato, or national Action? 
Yes l 

No 2 



'erlc 
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12'J 



r 

CONTINUED 



TVtatgj Untidy 
fGrcfc as many at apply.) 



1S4. Havo yog evar bnn given a special advantage or treatad unfairly because of yovr raca 
in any of the following situations? 

«wi Spacial Advantaojs 
(arclo at many at apply.) 

Gettingagood education , — > WgJ 1^"AB 

Getting a job. promotion or other work benefits 2 TfttfgA 2 TQ156BB 

rSiT" 1 ::::^ !&«»:::::::::::: «w" 

If so. please describe: TQ156E > 



157.- What aro your faolings about tho high school you graduated from? 

(Grclo ono numbor on oach lino.) 



Agrao 

Strongly 



Agrto 
Somtwhat 



Disagrao 
Soma what 



Ditagraa 

Strongly 



Dots not 

Apply 



School should have placed more emphasis on 
basic academic subjects (math, science, 

English, etc.) 1 2. 3 - 

School did not offer enough practical work 

experience 1 2 3 ■ 

School should have placed more emphasis on 

vocational and technical programs *. . -1 2 3 

?8chool'provided me with counseling that helped 

me find employment 1 2 3 

Scho8l should have given more attention to my 

needs as an individual ^. . . . 1 2 3 

School provided me with counseling that helped 

me continue my education 1 2 3 

Other comments about your high school 



.0 



15* 



/ 

The information you have given us in this questionnaire lots us know what you have been doing during tho past 
fc* years, particularly in Octobor 1975 and October 1974. This question *tk* about other time periods, so that wo 
will bo sure to have a complete picture of what you'vo boon doing since high school. 

Please road through all nine activities listed below, than for EACH time pariod circle tho number for EACH 
activity that you wero doing at that time. 
Grclo all that apply for EACH column. 



Oct. n 



Oct. 73 



L58AA-AE Working for pay at a full-time job 1 1 

L58BA-BE Working for pay at a part-time job 2 2 

L58CA-CE Enrolled in graduate or professional school 3 3 

L58DA-DE Taking academic courses at a two-year or four- 
year college * * 

158EA-EE Taking vocational or technical courses 5 5 

158FA-FE On active duty in the Armed Forces <or service 

academy) ■ 6 6 

158GA-GE Homefoaker 7 7 

158HA-HE Temporary layoff from work, looking for work. 

or waiting to report to work 3 8 

158IA-IE Other... 9 9 

Make sure you have circled at least one number in aach column.^. 



Oct. 74 

. 1 . 
. 2 . 
. 3 . 



What 
art you 

doing now? 



What do you 
txptct to 

bt doing in 
Oct. 1977? 



1 

2 

3 

4 

S 

6 
7 

8 
9 
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SECTION G: BACKGROUND INFORMATION 



Ploaso PRINT your name. 1 KWrtw, and th* telephone number where you can most usually bo roachod during tho coming 
year. 



YOUR NAME: 


TELEPHONE 


ADDRESS: 


AREA CODE 


NUMBER 


J 






CITY. STATE: 


ZIP: 


P1«a» PRINT ttw nam*, address and talaphen* numbtr of your parents. 


VOIITI PARFNTS' VAMP- 


TELEPHONE 


ADDRESS: 


AREA CODE 


NUMBER 








CITY: STATE" 


ZIP: 



Pfoaso PRINT tho namos and addrossos of two othor pooplo who will know where to got in touch with you during tho 
coming yoar. (List no mora than ono parson who now livts with you.) 



NAME 


TELEPHONE 


ADLXES3: 


AREA CODE 


NUMBER 


t 






CITY: STATE: 




ZIP: 




NAMF- 


TELEPHONE 


ADDRESS: 


AREA CODE 


NUMBER 


— — ^_ 






CITY: STATE: 




ZIP: 


Ploaso PRINT your spousa's full namo (if you aro married). 


SPOUSES FULL NAMP. 










Ploaso givo me following information about yourself. 








(a* Date of birth (month) 


(day) 


(yean 


<b) Sex; (Circle one.) Male 1 

Female 2 

(O Driver's License No. State 








(d) Whin did you complete this questionnaire? 


(month) 


(day; 


ivean 



THANK YOU FOR YOUR COOPERATION 

THIS INFORMATION WILL BE KEPT IN STRICT CONFIDENCE AND WILL BE USED ONLY FOR FUTURE 
FOLLOW-UPS IN THE NATIONAL LONGITUDINAL STUDY OF THE HIGH SCHOOL CLASS OF W2 
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Appendix D 

USE OF THE BALANCED REPEATED REPLICATION METHOD 
TO -COMPUTE THE VARIANCE OF IMPUTATION-BASED STATISTICS 
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Appendix D 

,„ _ THE BALANCED REPEATED REPLICATION METHOD 
TO COMPUTE THE VARIANCE OF IMPUTATION-BASED STATISTICS 

> 

The balanced repeated replication (BRR) method is a device for estimating 
the precision of estimates which come from surveys with complex sample de- 
signs. In this methodological study, BRR was used to estimate the variance of 
survey estimates when missing values in the data set were replaced using an 
imputation procedure. This section discusses the statistical theory underly- 
ing the BRR nethod of estimating variances and explains why applying the 
imputation procedure to the individual half samples before computing the 
half-sample estimates results in a variance estimate that accounts for * the 
added variation induced by the imputation procedure. The actual lmplementa- 
. tion of BRR in this investigation is also discussed. 

The balanced repeated replication (or balanced half-sample pseudoreplica- 
tion as it is sometimes called) was introduced by McCarthy (1966) as a method 
for estimating the variance of survey estimates, iiicluding the more complex 
statistics such as ratio estimates and regressidn coefficients for which 
analytical expressions for the variance are not readily available in the 
literature. The BRR method was specifically developed for the common survey 
design of two replicates per strata. In this situation, a total of 2. half 
samples may be formed with each half sample containing one replicate from each 
stratum where N is the number of, strata. For linear estimators, it has been 
demonstrated that if K half-samples ate independently selected from the entire 
set of 2 N possible half-samples, then the 'average squared deviation of the 
half sample estimates from the full sample estimate is equal in expectation to 
the usual variance estimate of the full sample statistic (McCarthy, 1966). 
Thus the variance of the fulL sample statistic |L may be estimated using the 
half-sample estimates (P^) by 

V.r(\) = X - Mj) 2 /'K . (D.D 

When all 2 N possible half-samples are used, this estimate of the variance will 
be equal to the usual full sample estimate of the variance. The reason for 
this is that the fluctuations among balf-sample variance estimates arises from 
between-stratum contributions to the estimates which are cancelled out when 



r 



all 2 possible half samples are used, McCarthy (1966) demonstrates that one 
may also eliminate this between strata contribution to the variance by con- 
structing a balanced set. of half-samples using orthogonal matrices such as 
those of Plackett and Burman (1946), The number of half-samples needed to 
achieve this balance will be a multiple of four and greater than or equal to 
the number of strata. When the sample design is based on a large number of 
strata (e.g., the sample design for NLS has 608 strata), it is not economical- 
ly feasible to use the large number of half samples that would be required (at 
least 608 for NLS). In this situation, it is possible to construct a set of K 
partially balanced half samples that will yield a more precise estimate of the 
variance than would have Seen obtained had K independently selected half-sam- 
ples been utilized (McCarthy, 1966). 

It should, be emphasized* that in using the BRR technique the average of 
the half sample estimates will be equal to the full sample estimate when a 
linear statistic is being computed. Similarly, the BRR estimate of the vari- 
ance of a linear statistic will be equal to the usual variance estimate. 
However, for nonlinear statistics such as ratio estimates of means and propor- 
tions which are the statistics being computed in this study, the average, of 
the balanced half sample means is not strictly equal to the full sample esti- 
mate so that no general claims ol unbiasedness can be made for the BRR vari- 
ance estimate. However, based upon the results of simulation studies, the BRR 
technique appears to yield relatively unbiased estimates of the variance which 
lead to robust statistical inference (Frankel, ]971). 

1 . Use of BRR to Estimate the Variance of Imputation-Based Statisti cs 

The balanced repeated replication estimators used in this study to ap- 
s proximate the sampling variance of imputation-based means and proportions have 

the same formal justification that would apply if no imputation were involved. 
When the primary sampling units (PSUs) are drawn from primary strata in pairs 
3nd with, replacement, statistics based entirely on a half -sample composed of 
one PSU from each stratum are independent of the companion statistics based on 
-the complementary half. Therefore, with independent, paired selections of 
primary units, the average of a half-sample statistic and its complementary 
estimate has a variance that is estimated unbiasedly by 'che squared difference 
of the companion estimators divided by four. With the weighting class and hot 
deck imputations made independently within each half sample and its comple- 
ment, this argument continues to hold. 
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To improve the precision of tnese pseudoreplication variance approxima- 
tions, the technique .for generating balanced and partially balanced sets of 
half-samples was developed by McCarthy (1966) so that by averaging K estimates 
of the form 



Var(p) = (Pjj. - M Ci ) 2 /4 (D-2) 

where is the estimate obtained using half-sample i and p Ci is the estimate 
obtained using the complementary half sample, one obtains a variance approxi- 
mation that is approximately V* times as precise as anyone of' the separate 
components. For linear sample statistics with no imputation involved, the 
average of a half-sample statistic and its complementary estimate is equiva- 
lent to the usual full sample estimate based on two PSUs per stratum (p) so 
that the half sample variance estimator described previously also applies 
rigorously to the full sample estimate. Further, the equation in (D.2) is 
equivalent to that given 1 in (D.,1). 

Unfortunately, the real utility of the BRR technique lies in estimating 
the variance of nonlinear statistics such as means and proportions ^where the 
average of half-sample and complement statistics are not equivalent to the 
corresponding full sample ratio. Second order Taylor Series approximations 
suggest that averages of half-sample ratios should be subject to roughly twice 
the estimation bias of corresponding full sample ratios . Therefore , to pre- 
serve the rigorous justification for BRR variance estimates, one should report 
the associated half-sample average along with the pseudoreplication standard 
error. Following the common practice of users of, BRR methods, we have ne- 
glected this rigorous theoretical justification to report means and propor- 
tions based on full sample ratios and full sample imputation. This reporting 
strategy can be empirically justified by noting that the relative difference 
between the full sample and half-sample statistics is small compared to the 
BRR relative standard error. 

2. Im plementation of the BRR Procedure to This Investigation 

" For this investigation, a total of 608 half samples (since there are 608 
strata in the NLS sample) would have been required to achieve full balance and 
eliminate all between-strata contributions to the variance which was impracti- 
[ cal since the processing off results from such a' large number of half samples 

would have been too costly \jhe following system .was used to construct a set 
I • 0 f 16 partially balanced half ' samples that eliminated some (but not all) of 
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the between-strata contribution to the variability of variance estimates. 

First, a set of 15 super strata were formed containing approximately the 
same number of strata using the following procedure. The strata weru sorted 
first according to cype of school (low versus high SES), then size of school, 
type of cbntrol (public versus ~jqy.vate) , geographic region, and finally, 
proximity to college or university. The first 300 strata were regions with 
low socioeconomic status (SES), and the second 300 were regions with high 
socioeconomic status. The final eight strata were composed of schools which 
had not been listed on the frame for the base-year sample. In creating the 
super strata, 7 were formed from the first 300 strata with low SES. The 
sorted group of low SES strata were partitioned so that the first 43 strata 
were designated as super stratum number 1, the second 43 strata as super 
stratum . number 2, and so on until ^the seventh super stratum contained the 
remaining 42 strata from the low SES group. The remaining 8 super strata 
were selected from the 300 strata with high SES and the 8 strata of supple- 
mental schools in a similar manner with the 15th super stratum containing the 
last 31 strata with h, *h SES from the sorted file and the 8 'strata containing 
supplemental schools. 

These 15 s>uper strata were thetj used in the BRR procedure to create the 
16 half samples. Since the BRR procedure is designed for two replicates peir 
stratum, the following adaptation of the procedure was needed for this inves- 
tigation. Most of the NLS strata contain two schools, but a few have only one 
school due to nonresponse. Also; some schools would not participate in the 
Base Year Survey and so substitutions were made for \these schools. Subse- 
quently, some students from these schools which ^id not participate in the 
\ Base Year Survey were included in the follow-up surveys so that some of the 
NLS strata contain three or four schools. To partition the schools into" two 
replicates, the file was first sorted by final stratum and then by school. 
Within each super stratum, the schools were numbered as they occurred in the 
file. Within each super stratum, the even numbered schools were, regarded as 
the first "replicate" and the odd numbered schools as the second "replicate." 
•A balanced repeated replicate design for 15 strata, which required 16 half- 
samplei, was then used where all of the even numbered schools from a super 
stratum were included in the half sample if the BRR design specified that the 
""first replicate from that stratum was to be used and the odd numbered schools 
otherwise. 

v> o 
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STDERR: STANDARD ERRORS PROGRAM 
FOR SAMPLE SURVEY DATA 
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STDERR: STANDARD ERRORS PROGRAM FOR SAMPLE SURVEY DATA 



An RTI computer program (STDERR) was used to estimate the proportions, 
means, and their sampling errors for the 20 selected critical items and do- 
mains. Since the sample size for each^domain was not fixed, it was necessary 
to compute the estimated proportions, P d> and means, fij, as the ratios of two 
random variables. 

For the ratio estimate of a proportion, the numerator estimated the total 
number of students in domain d who would have chosen the particular response 
option and the denominator estimated the total number of students in the 
domain. Explicitly, 

L a h b hi 



where 



P„ = 100 



i ,1, I 



L a h b hi 

A A A 



loo xai 



D(d) 



L = number of final strata 



a = number of sample schools in final stratum-h 
h 

b = number of sample students in school-hi 
hi 

W .. = nonresponse adjusted weight for student-j in school-i 
niJ from stratum-h 

1 if student-hij belonged to domain-d 

D. 



•! 



hij 

0 otherwise 
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1 if student-hij answered the questionnaire item in a 
specified manner* (e.g., yes) 



0 otherwise 



Y(d) = estimated number of students who answered the question- 
naire item in a specified manner for domain-d. 

D(d) = estimated number of students in domain-d. 

The standard error of was estimated by the square root of the variance of 
P d , where (Woodruff, 1971 and Cochran, 1977) 

,2 



.2 



' h=l h=l 



a h t Z hi + (d) " Z h. + (d) l' 



Var(P d ) = (100)' \ I a h I (a h -l) 



+ J V a h tz hi ,(d) - z h ^(d)] 2 

1 T~ a h 1 T a~^l) 

h=M+l % Q i=l X \- L} 

i 



L a, a h /B..-b.. 
h=M+l *h i=l I hi ' 



b hi' [Z h ..(d)^ h .,(d)] 2 | 
1 j=l C^-l) J 



and 



M = total number of final strata with schools selected with 
probabilities proportional to size 



= total number of schools in sampling frame for final 
stratum-h * 



B, . = total number of senior students enrolled in school-hi 
hi 

Z hi ,(d) = W hi j(Y hi j-P d )/D(d) = weighted Taylorized deviation 



( 



b hi 



Z^ + (d) = I Z^(d) = weighted deviation totals by school 



9 
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•h Z. ..(d) * 

2 (d) s I -==- s average school totals by stratum 

h * + i=l *h 

Z. ..(d) - 

7 ( d ) - Jlil = average weighted deviations by school 

^ b hi 

L a h b hi 

° (d) " * .?, .5. ■ w hij d wj • 

h=l 1=1 j=l r 

The equation for Var(P d ) assumes, without loss of generality, that the final 
strata have been reordered so that the first M are those in which schools were 
selected with unequal probabilities proportional to size (PPS). The. first 
term in the equation involves the between cluster estimate of variance for the 
PPS strata. The second term involves between cluster estimate of variance 
us£ng the first stage finite population correction factor for the strata where 
schools were selected with equal probabilities, while the third term adds back 
the appropriate proportion of the within school variance for the equal proba- 
bility strata. This equation gives an unbiased estimate of the variance for 
the equal probability final strata and' a slight overestimate of the variance 
for the PPS strata. Also, it should be noted that Z hij (d) is the Taylorized 
deviation of the ratio estimate P d - Such approximations are most valid for 
large samples. 

The estimated mean for domain-d, is also obtained as the ratio of 

two random variables. Explicitly, 

L a^ b hi 

* * * "hij "hij *hij 
h=l 1=1 3-1 

•d* L b M 

M rt J-l hlJ hlJ 

where Y... represents the value of the quantitative variable under considera- 
tion fo/student-hij and the other parameters are as previously defined in the 
estimation formulas for proportions. The standard error of m d was estimated 
by the square root of the variance of « d . The variance estimator for m d has 
the same form as that presented for the f ample proportion, P d> with m^replac- 
9 ing P in the definition of the Taylorized deviation Z hij (d). 
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Appendix F 

COMPARISON OF NO IMPUTATION ESTIMATES WHEN 
INCONSISTENT DATA ARE REMOVED XNIC) AND WHEN RETAINED (NI) 
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Glossary of Terms Used in the Tables 

SAMPLE SIZE - number of sample members eligible to respond to a particular 
item for the domain under consideration. 

Y-TRUE * the estimate obtained using the telephone corrected and com** 

pleted data. 

NI - estimates obtained using no imputation or editing procedure on 

the experimental data set. 

NIC - estimates obtained using no imputation or editing procedure on 

the experimental data set after inconsistent data are removed. 

ME - measurement error caused by the use of data containing logical 

inconsistencies . 

jf 

RB% - the^ relative bias defined to be the bias divided by the value 

of Y-TRUE, expressed as a percentage. 

BR - the bias ratio defined as the bias divided by the standard 

error of the estimate. 

RVMSE* - the relative root mean square error defined as the square root 

of the mean square error divided by the value of Y-TRUE and 
expressed as a percentage. 
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Comparison of no imputation estimates when Inconsistent data are removed (NIC) and when retained (NI) for 
proportions estimated for the total population 
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-0,02 


•0,07 


0,16 


0.26 


0,09 


0.07 


3.36 


3.91 


0,02 


« 0,10 


0,00 


0.06 


0.00 


0.01 


3.95 


3.95 


0,00 


0,00 


0,00 


-0.01 


0.00 


-fc.oi 


1.06 


1.06 


0,00 


0,00 


0,37 


0.39 


0.06 


0.06 


5.71 


5.72 


0,00 


0,00 


-0.19 


-0,16 


-0.02 


-0.02 


7.76 


7.76 j 


0,01 


0,09 


•0.39 


-0.93 


1 -6.il 


-0.12 


3.52 
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Comparison of no imputation estimates when inconsistent data are removed (NIC) and when retained (NI) for 
proportions estimated for males 



HCfl 
TttlC 

mo 

TU16 
1U9A 
K**9C 
FU90 
,T«9G 
FU10 
FttiO 
lUiO 
FU10 
FV12 
*U12 
FU12 
F«29 
|U29 
F«33 
TU33 
F033 
FQ5* 
7 FU51 
FU52 
TU52 
1U66 
FU66 
FUC6 
FU90 
TO90 
fU90 
1U90 
TO101 
1U101 
TU102 
FU102 

Fona 
Fuiie 

Fttlltt 

FM129 

FU129 

FU129 

FW129 

FU131A 

TU131C 

FU1310 

F0131F 

TU136B0X 

FU136A 

FU136C 

FU136D 

ERLC 



RESPONSE 
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•SAMPLE 




ME 


HE 


NI 


NIC 


NI 


NIC 


NI 


NIC 


SIZE 


- 

V-tFRUE 


BIAS 


HB% 




R8JJ 


BR 


BR 


rvmse* 


RVrtSE* 


1357 


77*00 


-2,51 


-3*25 


• 3,26 


0,16 


•5* 36 


0*76 


3.33 


0.79 


7357 


19,6*1 


-0,13 


• 0,65 


-0,55 


•9,02 


•0*17 


-3*17 


3.21 


9*36 


7357 


4 ,63 


0,56 


12,10 


12,31 


-6,11 


2*67 


•1.27 


13.17 


6.|9 


* 7357 


9,26 


-0,91 


-9,60 


-9,86 


•9,03 


-1*61 


•U39 


11.21 


141.65 


7357 


71*16 


•1*16 


-6,291 


-6,31 


-0,97 


•5*21 


•0*72 


6.16 


1.66 ' 


7357 


35,17 


-0,13 


-1*22 


-1,31 


•9,09 


•0*59 


•5.91 


2.63 


9.39 


7357 


1,50 


0,07 


1,55 


1 * 69 


•17,61 


0,52 


-3»99 


1.09 


16.36 


7357 


7,12 


-1*39 


-19,52 


-19,63 


-16,90 


-3.*3 


•3*06 


20.56 


19.89 


7357 


66,57 


0,01 


0,01 


0,00 


2, 19 


0*00 


3.06 


0.72 


2.31 , 


7357 


11,75 


-0,05 


-0,12 


-0,67 


-11 ,79 


-0*15 


•3.62 


1*35 


15.29 


7357 


1 ,39 


•0,01 


• 0,71 


-1 ,67 


-5,56 


-0* 12 


-0.39 


13.51 


15.13- 


7i57 


16,28 


0,07 


0, 36 


0,55 


1 ,66 


0* 22 


0.61 


2.59 


3*21 


< * *4 OK 

4 1273 


36, 19 


0,60 


2,21 


1 ,93 


-9,61 


ft At At 

0 • 11 


•2.31 


1.77 


I0o16 


1Z73 


6,96 


0,09 


1,26 


2,11 


5,96 


0* 25 


0*67 


6*56 


10.62 


1273 


56,63 


• -0,69 


• 1*56 


• 1,19 


5,10 


-0*19 


1.76 


3. 37 


6.20 


c n c * 
6U56 


90,26 


0,03 


■0,03 . 


0,08 


0,13 


0*23 


0.36 


0. 36 


0.39 


6058 


9,7** 


-0,03 


-0,30 


-0,77 


-1*27 


•0*23 


-0.36 


3. 36 


3.70 


1633 


21,65 


1,62 


6,10 


7,50 


•17*66 


1*17 


•3.10 


9.07 


16.11 


1633 


3,90 


0, 17 


1 , 36 


5,11 


5*to6 


0* 31 


0.11 4 


16.86 


11.62 


w ii 
1633 


71 .15 


-1,99 


-2,67 


•2,16 


1* 61 


-1 *11 


2.76 


3.00 


5.11 


7357 


13.76 


0«16 


1* 09 


0,53 


•0*61 


0*27 


-0*35 


1.97 


1.93 


#357 


56,2*1 


-0,18 


-0,65 


• 0,11 


0*50 


•0*27 


0.35 


1 *51 


1.50 


3970 


15*61 


0,22 


0,17 


0,50 


•0* 18 


0*21 


•0.07 


2.16 


2.32 


3970 


■ 51,16 


-0,22 


-0,10 


-0,13 


0* 15 - 


-0*21 


0.07 


1.62 


1.96 


$970 


16*91 


0,17 


2*16 


1,97 


•1 *96 


0* 77 


-0.71 


3.23 


3.36 


3970 


35,36 


1,13 


3,19 


3,95 


3*62 


1*61 


1.21 


1.65 


1.61 


1Q1« 

3970 


H 5,66 


-1,60 


-3*50 


•3,66 


-1 *99 


-1 *69 


•0.65 


1.51 


3.06 


3970 


66,03 


0,23 


0,31 


0, 32 


0* 72 


0*29 


0.66 


1.16 


1.31 


3970 


1*61 


0, 30 


6,16 


6,26 


-0* 76 


0*16 


-0*06 


11. 36 


11.63 


3970 


5,22 


} -0,01 


-0,19 


-0,61 


-2*00 


-0*06 


•0.23 


9.72 


6.78 


3970 


21*11 


-0,51 


-2,11 


-1,97 ■ 


-1*11 


-0*60 


•0.13 


3*63 


3.51 


7357 


61 ,67 


0,00 


0,00 


-0,06 


-0 • 11 


•0*10 


•0.11 


0.77 


0.77 


7357 


15.13 


0,00 


0,00 


0 ,16 


0*63 


0,10 


0.11 


1* 36 


1*35 


1031 


61 ,36 


-0,53 


-0,66 


-0,30 


0*57 


-0* 06 


0.11 


3.61 


1*03 


1031 


36*61 


0,53 


1*37 


0,16 


-0*91 


0 * 06 


•0.11 


5.77 


6*10 


7357 


67,00 


0,00 


0,00 


-0,03 


0* 01 


•0*06 


0.02 


0 .12 


0*42 


7357 


1,56 


0,00 


0 o 0 0 


-0 ,66 


-3« 63 


-0,05 


•0.31 


12.61 


12,93 


7357 


11*11 


0,00 


0,00 


0, 35 


0*13 


0, 09 


0.11 


3«97 


3*99 


7357 


9,93 


0,03 


0, 30 


• 0 ,52 


-0*65 


-0, 06 


»0.12 


5*25 


5*23 


7357 


51*11 


0,00 


0,00 


0,02 


0*03 


0,01 


0*02 


a.70 


1*70 


7357 


2,62 


-0,01 


-0* 35 


-0,16 


-0*09 


-0, 02 


-0.01 


6.67 


6*69 


7357 


32,61 


-0,02 


-0*06 


0,07 


0*11 


0.03 


0.07 


2.02 


2*01 


2592 


55,12 


0,09 


0*16 


• 0,01 


•0*21 


-OtJl 


•o.ie 


1.61 


1.71 


2592 


7. Hi 


0,00 


o.ou 


0,25 


0*56 


0*02 


0*06 


6.66 


6.79 


2592 


2,50 


0,00 


0,00 


* 0,25 


0*56 


0*01 


0*02 


23.10 


23.17 


2592 


62,05 


-o,u 


-0,17 


0,03 


0*31 


0*02 


0*20 


1.13 


1.53 


2617 


20,73 


0,00 


0,00 


0,00 


0*00 


0*00 


0.00 


3«56 


3*56 


2231 


51,63 


0,00 


0,00 


-0,03 


•0*03 


-0*0|L 


-C.02 


2.00 


2.00 


2231 


6,72 


0,00 


0,00 


0,62 


0*65 


0*06 


0.06 


7.11 


7.15 


2231 


2,32 


0,00 


0,00 


0,62 


0*65 


0*03 


0.03 


16.75 


16.76 


2231 


53,11 


0,00 


0,00 


-0,21 


•0*21 


-0*10 


•0.09 


2.26 


2.27 


















- ' < 1-4 


3- ■ 



Comparison of no imputation estimates when inconsistent data are removed (NIC) and when retained (MI) for 
proportions estimated for individuals of low socio-economic status 



RESPONSE 



ERIC 



SAMPLE 

size 

•1220 
•1220 
•1220 
•1220 
•1220 
.•1220 
W220 
•1220 
•1220 
•1220 
• *220 
•1220 
1069 
1069 
1069 
3150 
3150 
1099 
10» 
1099 
•1220 
•1220 
1*20 
1*20 
1*20 
1H20 
1*20 
1*20 
1*20 
1*20 
1*20 
-*220 
*220 
230 
230 
*220 
*220 
*220 
*220 
*220 
*220 
*220 
2093 
2093 
2093 
2093 
235* 
2021 
2021 
2021 
2021 



f-TRUt 

69,91 
10,77 

3,6* 
11, *7 
66,65 
16,26- 

5,66 
10,13 
62,66 

9,10 

1,93 
26,09 
30,32 

7,30 
62*39 

91, *6 
6,5* 
26,92 
3,71 
69,36 
6*, 63 
35,17 
*9,76 
50,22 
30,72 
29,91 
39,37 
52,36 
6,** 
6,32 
52,66 
9*, 03 
5,97 
47,12 
32,66 
90,23 
l.W 
6,33 
9,35 
35,96 
6,12 
*6,57 
71,01 
5,12 
3,97 
26,91 
1*,1* 
71,61 
5,72 
3,65 
21, *6. 



HE 
BIAS 

• 1,06 
•0,06 

0,*2 
-1,*7 
-2,33 

• 0,79 
<U22 

-2,03 
-0,03 
0,03 
0,07 
-0,07 
1,26 
-0,10 
-1,19 
-0,01 
0,01 
1,09 
0,53 
-1,** 
0,26 
-0,26 
0,*5 
-0,*5 
0,63 
0,66 
-1,51 
0,29 
0,*5 
0,17 
-0,91 
0,00 
0,00 
-2,01 
2,01 
0,00 
0,00 
0,00 
0.00 
0,00 
-0,0* 
0,0* 
0,2* 
-0,06 
0,00 
•0,0* 

b.oo 

0,00 
0,00 
0,00 
0,0* 



ne 

KB* 

-1,51 
r0.55 
10,92 ' 
-12,61 
-3,36 
•*.65 
6,01 
•20*05 
-0.0* 
0.5* 
3,61 
•0,26 
*,22 
•1,37 
•1,90 
-0,01 
0,11 
*,0* 
6,90 
-2,07 
0,*3 
-0,79 
0,90 
-0,69 
2,70 
2,27 
-3,63 
0,53 
6,96 
2,0* 
-2,76 
0,00 
0,00 
-2,99 
6,11 
0,00 
0,00 
0,00 
0,00 
JO. 00 
-0,65 
0,06 
0,33 
w*.56 
0,00 
-0,1* 
0,00 
0.00 
0,00 

0*00 

0,16 



8R« 

-1,*7 
• 0,50 
.11,0* 
•12,77 
-3,*6 
.*,*6 
6,66 
-19,92 
-0,13 
1,03 
*,17 
•0,35 
*,52 
-0,73 
*2,*1 
0,00 
0,09 
5,71 
10,93 
.2,02 
0,0* 
.0,09 
1,20 
.1,19 
2,5* 
5,*5 
.*,*3 
~0,01 
6,9* 
2,52 
.1,97 
.0,02 
0,36 
.1,02 
2,10 
.0,1* 
5,51 
0,9* 
.0,57 
-0,17 
-0,29 
0,2* 
0,19 
-1,5* 
.0,56 
0,10 
0,00 
.0,06 

0,14 
• 1,75 
0,55 



NIC 
RBX 

1>25 
-17,95 
-15,62 
-11,05 

0*69 
•16,69 
•16,56 
•20,59 

1,06 
-7,17 
•5,59 

0.5' 
-10,27 

5,*7 

*,55 
-0,02 

0,23 
-19,32 

9,16 

7,00 
-0,16 

0,55 

0. 75 
-0,7* 

1. H6 
2,59 

-2,96 
0,60 
•1,26 
-0,61 
-0,55 
•0,05 
0,*9 
2,15 
-*,36 
•0,15 
3,59 
1,01 
-0,29 
-0,21 
-0,21 
0,25 
0,07 
•0,96 
0,00 
0,05 
0,00 
-0,09 
0,25, 
•1,65 
0,25 



HI 

BR 

-1,2* 
•0,07 
1,29 
•2,92 
•2,59 
•0,71 
0,69 
-2,75 
-0,09 
0,25 
0,56 
-0,11 
1,22 
-0,06 
-1,01 
-0,01 
0,01 
1,06 
0,96 
•1,57 
0,05 
-0,05 
0,55 
-0,55 
0,*6 
0,71 
-1,09 
0,00 
0,57 
0.*0 
-0,55 
-0,0* 
0,0* 
-0,1* 
0,1* 
-0,25 
0,16 
0,15 
-0,05 
-0,06 
•0,05 
0,15 
0,09 
-0,06 
-0,05 
0,02 
0,00 
•0,05 
0*01 
•0,17 
0,06 



NIC 
BR 

1,05 
-5,*0 
•1,*6 
•2,57 
0,5* 
*5,39 
-2,96 
•5,52 
0,76 
•1,60 
-0.55 
0,11 
-2.69 
0.*5 
2.21 
•0.03 
0.05 
•*,1* 
\0,57 
N3.02 
•0,11 
0,11 
0,20 
•0.20 
0,29 
0.5* 
•0.76 
0.15 
•0.10 
•0.06 
-0.09 
-0.05 
0.05 
0.29 

• 29 

• 2* 
,16 

• 1* 

• 05 

• 09 

• 02 

• 15 

• 05 

• 06 

• 00 

• 01 

• 00 

• 05 
0«02 

•0«16 
0*0* 



•0« 



•0« 



-0« 



NI 

fttfKScx 

l.*S 
6a 66 
15,95 
13«*9 
3,71 
7*70 

10a01 

21.20 
1.33 
*.*9 

12.31 
5.09 
5.56 

11.59 
2.65 
0.5* 
5.61 
5.06 

15.71 
2.50 

I. 57 
2.90 
5.61 
5.77 
5.56 
5.69 
5.99 
*.*1 

15.66 
6.77 
6.17 
0.56 
6.9* 
7.56 

15.02 
0.65 

19.7* 
7.26 
7,51 
2,19 
9,05 
1,56 
2,05 

15,50' 

II, 7* 
5,07 
6,99 
1,59 

10,1* 
10,16 
5,21 



Nit 

rVasTx 

1,72 
16*71 
15,66 
11,97 
1,** 
17,55 
19,57 
21,50 
1.72 
6.12 
11.55 
5.06 
. 10.67' 
13.10 
*.77 
0.55 
5.91 
19.67 
lft.*5 
7.56 

I. 65 
3.0* 
5.7* 
3.70 
5.16 
5.01 
*.6l 
41.55 

12.25 
7.65 
5.65 
0.56 
6.95 
7.5* 

15.59 
0.63 

19.76 
7.29 
7.32 
2.22 
9*06 
1.5* 
2.07 

15.61 

II. 76 
5.05 
6,99 
1,60 

10,1* 
10,16 
5,15 
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Comparison of no imputation estimates 
proportions estimated for individuals 



when inconsistent data are removed (NIC) 
of h,igh ability 



and when retained (NI) for 



SAMPLE 

!lt« RESPONSE SIZE 

TUlA 1 2839 

TU1C 1 2639 

TU10 I 2839 

TU1G 1 2639 

TU9A 1 2859 

TU9C 1 2839 

TU9U 1 2639 

f U9S 1 2639 

TU10 1 2839 

IU10 2 2639 

TU10 3 2639 

TU10 4 2839 

TUI2 1 732 

TU12 2 732 

TU12 3 732 

TU29 1 2107 

TU29 2 "107 

TU33 1 993 

TU33 2 993 

i TU33 3 993 

CO TU51 1 2839 

? TU51 2 2639 

TU52 1 2155 

TU5 - 2 2155 

TU66 X 2155 

TU66 2 2155 

TU66 3 2155 

TU90 1 2155 

TU90 2 2155 

XU90 3 2155 

TU90 4 2155 

TUlOl 1 2839 

TU101 2 2839 

TU102 1 951 

IU102 2 951 

TUlld 1 2839 

T0116 2 2839 

T011IT 3 2839 

10129 1 2839 

TU129 2 2839 

TU129 3 2839 

IU129 H 2839 

TU131A 1 *67 

TU131C 1 *67 

fUlilO 1 867 

IU131F % 1 8 67 

TU136B0X 1 9«*0 

IU136A 1 731 

T0136C 1 731 

TU1360 1 731 

TU136F 1 731 

o 151 



ME ME 

Y-TRUE BIAS RB* 

71. 7H -2.99 -H.16 

26.28 -0.03 -0.11 

H. ll 0.06 1.H5 

6.38 -0.97 -11.57 
*3.06 -9.13 -14. H7 
58.79 -0.H8 -0.81 

2.66 0.01 0.34 

4.91 -1.65 -33.62 

53.65 -0.12 -0.22 

20. HO -0.05 -0.24 

I. 26 -0.0H -3.18 
24.70 0.20 0.60 
22.03 1.07 H.65 

6.26 0.08 1.27 

71.72 -lvl6 -1.61 

69.62 0.00 0.00 

10.16 0.00 0.00 

9.39 1.63 17.36 
2.35 0.18 -7.6H 

68.26 -1.81 -2.05 

21.90 0.32 1.+6 

76.10 -0.32 -0.HO 

H7.36 0.05 0.10 

52.62 -0.05 -0.09 

11.65 -0.2? -2.31 
32. HO 0.98 3.02 
55.95 -0.T1 -1.26 
62.00 0.00 0.00 

5.15 0.09 1.74 

5.26 0.00 0.00 

27.59 -0.09 -0.32 

6H.32 0.00 0.00 

35.66 0.00 0.00 
60.71 -0.18 -0.29 
39.29 0.18 0.H5 
9H.H3 0.00 0.0C 

o.m o.oo o.oo 

5.17 0.00 0.00 

10.73 0.0H 0.37 

58 . OH 0.00 0.00 

2.37 0.00 0.00 

28.86 -0.0H -0.13 

76. H6 0.00 0.00 

12.63 0.00 0.00 

4.61 0.00 0.00 

22.9H 0.00 0.00 

22.20 0.00 0.00 

72.05 0 t 00 0.00 

17.65 0.00 0.00 

4.05 0.00 0.00 

17.55 0.00 0.00 



NI NIC NI 

rbx ho* ait 

-4.23 -0.15 -3.01 

0.03 -5.55 0.00 

1.53 -7.26 0.15 

-11. H2 -10.20 -2.02 

-1H.60 -5.13 -10.55 

-0.73 -H.H5 -0.39 

0.38 -3.65 0.02 

-33.39 -32.74 -3.50 

-0.30 2.76 -0.13 

-0.52 -10. 5H -0.16 

-2.35 -17. 9H -0.11 

1.21 3.56 0.27 

H. 96 -11. 9H 0.64 

I. 47 4.26 0.06 
-1.65 3.29 -0.60 
-0.03 0.02 -0.04 

0.32 -0.25 0.04 

17.85 -35.65 1.29 

7.94 5.49 0.53 

-2.11 3.64 -1.42 

1.49 0.11 0.36 

-0.41 -0.03 -0.36 

-0.07 -0.84 -0.03 

0.06 0.75 0.03 

-2.60 -4.66 -0.43 

3.77 2.63 1.04 

-1.64 -0.66 -0.90 

-0.07 0.66 -0.04 

2.49 0.02 0.13 

-0.16 -2.19 -0.01 

-0.26 -1.13 -0.05 

0.05 -0.02 0.02 

-0.09 0.05 -0.02 

0.25 0.54 0.14 

-0.39 -0.84 -0.14 

-0.02 0.01 -0.05 

-13.67 -24.49 -0.60 

1.53 1.69 0.20 

0.03 -0.30 0.00 

-0.08 -0.03 -0.04 

0.33 0.37 0.02 

0.11 0.16 0.03 

-0.05 -0.05 -0.02 

0.21 0.21 0.02 

0.21 0.21 0.00 

0.21 0.21 C.01 

0.00 0.00 0.00 

-0.06 -0.06 -0.01 

0.16 0.16 0.01 

0.16 0.16 0.00 

0.16 0.16 0.01 



nic vNi 

BK RVR5E* Rtf*SE* 

-0.09 H.H6 1.69 

-£.07 3.H9 6.16 

-0.76 9.92 11.74 

-1.60 12. 7H 11.67 

-2.73 14.67 5.H6 

-2.55 2.01 4.76 

-0.32 13.60 12.37 

-2.62 3H.72 34.72 

1.14 2.21 3.69 

-4.10 2.64 10.65 

-0.92 20.94 26.40 

0.75 H.56 5.91 

-1.52 9.14 14.30 

0.25 17.27 17.46 



1.04 


3.16 


4.55 


0.03 


0.77 


0 .77 


-0.03 


6.86 


6.66 


•3.43 


22.57 


37.13 


0.37 


16.60 


15.53 


3.10 


2.57 


3.62 


0.02 


4.34 


4.14 


-0.02 


1.21 


1 • 16 


•0.37 


2.20 


2.41 


0.37 


1.96 


2.17 


•0.73 


6.50 


7.69 


0.75 


5.21 


la 11 

4 • r 1 


-0.37 


2.44 


1.69 


0.37 


1.84 


1.96 


0.00 


16.60 


16.56 


• 0.19 


12.30 


11.76 


-0.24 


4.62 


4.72 


-0.01 


2.05 


2.06 


0.01 


3.71 


3.72 


0.29 


1.76 


1.91 


-0.29 


2.76 


2.95 


0.03 


0.42 


0.41 


-1.12 


26.50 


32.61 


0.22 


7.56 


7.65 


-0.04 


6.75 


6.77 


-0.02 


1.76 


1.79 


0.03 


11.63 


11.63 


0.04 


3.26 


3.26 


-0.02 


2.12 


2.12 


0.02 


10.32 


10.32 


0.00 


22.54 


22.54 


0.01 


11.15 


11.15 


0.00 


6,99 


6.99 


-0.01 


3.40 


3.40 


0.01 


9.61 


9.61 


0.00 


26.85 


26.65 


0.01 


11.34 


11.34 
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Comparison of no imputation estimates when inconsistent 
proportions estimated for blacks 



data are removed (NIC) and when 



retained 



(NI) for 



i ten 



RESPONSE 



to 
I 



mm 

1U1C 
IU10 
1416 
1U9* 
IU9C 
TUW 
TU96 
T«10 

JtllQ 

mm 

IUI2 
IU12 
1U|2 
I«29 
I«29 
I«33 
IU33 
IU33 
TU31 
IU51 
1U52 
IU32 
fU66 
fti66 
TU66 
IW9Q 
1U90 
VU90 
IU90 
IU101 
1U191 
IU102 
IU102 
TU116 

mtio 
inns 

IU129 

IU12* 

IU129 

?U129 

I«13?* 

10131C 

1U131U 

IU131F 

IU136B0X 

fUl364 

1U136C 

ERIC 153 



KAflPLE 




ME 


HE 


NI 


SUE 


t-TRut 


BUS 


R8X 


R8X 


1963 


69/37 


-l.*5 


-2.09 


• 1.96 


1963 


17.03 


-0.27 


• 1.56 


-1.63 


1963 


5.51 


•0.1* 


-2.53 


•2,36 


1963 


1*.*5 


•2.3* 


•16.19 


-16.9* 


1963 


6* ,93 


-2.93 


•*.51 




1963 


26.71 


-1.56 


• 5.6* 


•5,16 


1963 


6.63 


0.15 


2.19 


2,60 


1963 


13.61 


-3.*1 


-2*.66 


-25,26 , 


1963 


59.55 


0.57 


0.95 


0.69 j 


1963 


12.36 


-0.3* 


•2.75 


-1.1* / 


1963 


1.65 


0.11 


5.93 


6,29 ' 


1963 


26.2* 


-0.3* 


-1.29 


-2,06 


505 


36.67 


1.66 


*.76 


*,59 


505 


12. *5 


-0.55 


•*.*1 


• 0,61 


505 


*6.66 


-1.31 


-2.69 


-3.51 


1*55 


92.03 


0.00 


0.00 


-0.05 


1*55 


7.97 


0.00 


0.00 


0.62 


569 


37.65 


0.66 


1.7* 


1.1* 


569 


5.60 


1.06 


16.60 


20,90 


569 


56.35 


•1.7* 


-3.06 


-2,92 


1963 


50.21 


• 0.76 


-1.55 


-0,11 


1963 


*9.79 


0.00 


0.00 


0.11 


926 


4*. 9* 


1.55 


3.** 


*.01 


926 


55.06 


•1.55 


61 


.3.27 


926 


23.06 


1.62 


7.06 


6.01 


926 


35.77 


0.76 


2.12 


2.31 


926 


H.15 


•2.56 


•6.26 


•6.50 


926 


*2,92 


0.32 


0.7* 


-0.10 


926 


6.37 


1.62 


26.57 


30.06 


9?6 


10.97 


0.17 


1.55 


•0.72 


926 


39.75 


•2.32 


-5.63 


-*.50 


1963 


90.16 


o.oo 


0.00 


-0.30 


1963 


9,6* 


0.00 


0.00 


2.61 


160 


6*. 15 


-1.26 


• 1.96 


-1.97 


iao 


35.65 


1.26 


3.51 


3.53 


1963 


39.0* 


0.00 


0.00 


•0.35 


1963 


1.2* 


0.00 


0.00 


1.25 


1963 


9.72 


0.00 


0.00 


3*07 


1963 


12.11 


0.00 


0.00 


-0.21 


1963 


51.15 


0.00 


0.00 


0.02 


1963 


H.2* 


0.00 


0.00 


1.72 


/1963 


32.50 


0.00 


0.00 


-0.17 


6** 


73.5* 


0.00 


0.00 


•0.03 


6** 


6.67 


0.00 


0.00 


0.99 


6* * 


*.02 


0.00 


0.00 


-1.6* 


6** 


2*.*2 


0j25 


1.02 


1.55 


7*5 


17.27 


0.00 


0.00 


0.00 


615 


73.** 


0.00 


0.00 


0.25 


61$ 


6.5* 


0.00 


0.00 


0.90 


615 


•••76 


0.00 


0.00 


-1.6* 


615 


16.05 


0.16 


0.66 


-0.55 



NIC 


NI 


NIC 


NI 


NIC 


RDS 


BR 


8R 


ft\fn5E« 


RVflSEX 


2.27 


•1.00 


1.13 


2.61 


3.03 


•13.76 


-0.35 


•2.66 


5.*0 


1*.72 


•17.67 


-0.19 


•2.09 


12.31 


19.56 


•1*.06 


-2.10 


•1.62 


16.76 


16.52 


1.61 


-2.13 


0.62 


*.90 


2.53 


-1*.06 


-0.77 


•2.90 


e.*3 


t*.69 


•16.26 


0.27 


•1.36 


10.52 


20.1* 


-22.9* 


-3.31 


•2.95 


26.36 


2*. 22 


2.15 


0.*3 


0.90 


2.26 


3.19 


-9.79 


-0.19 


-2.11 


5.67 


10.6* 


6.23 


0.29 


0.20 


29.66 


31.66 


-0.71 


-0.*6 


•0.1* 


*.97 


*.91 


-10.61 


0.6* 


•l.*6 


6.*7 


1^79 


12.32 


-0V03 


0.67 


16.0* 




5.32 


-0>6* 


0.96 


6.*7 


17.59 


-0.0* 


-O.fc 


-0.05 


0.66 


Nj.66 


0.56 


0.06 


0.05 


10.16 


10^1 


•21.01 


0.22 


•*.23 


5.1* 


21.56 


1*.65 


0.62 


0.66 


32.77 


22.05 


12.60 


-0.95 


3.63 


*.23 


13.02 


-1.1* 


•0.03 


-0.35 


3.26 


3.*7 


1.15 


0.03 


0.35 


3.29 


3.50 


1.93 


* 0.9* 


0.39 


5.65 


5.29 


•1.56 


-0.9* 


•0.39 


*.76 


*.32 


0.00 


1.05 


0.00 


11.05 


7.67 


*.91 


0.*2 


0.95 


5.9* 


7.13 


-*.27 


-1.65 


•1.96 


7.60 


5.66 


2.22 


-0.02 


v 0,115 


*.36 


5.36 


2.57 


l.*3 


M 0.16 


36.69 


1*.5* 


-3.37 


•0.06 


-0.25 


12.0* 


13.** 


-1.66 


-0.67 


-0.30 


6.0* 


6.50 


-o.*o 


-0.27 


•0.36 


1.1* 


1.19 


3.75 


0.27 


0.36 


10.51 


10.95 


0.00 


-0.31 


0.00 


6.50 


7.22 


0.00 


0.31 


0.00 


11.6* 


12.93 


•0.21 


-0.32 


•0.20 


1.13 


1.09 


1.51 


0.0* 


0.05 


26.1* 


26.26 


1.79 


0.32 


0.19 


9.63 


9.36 


•0.79 


-0.02 


•0*09 


6.60 


6.61 


-0.06 


0.00 


•0.02 


2.29 


2.25 


2.03 


0.1* 


0.17 


11.66 


11.62 


0.13 


-0.0* 


0.03 


* .25 


*«23 


-0.25 


0.00 


• 0.06 


3.66 


3.60 


1.99 


0.05 


0.11 


16.62 


17.09 


-0.67 


-0.07 


•0.03 


25.63 


25.76 


l.*7 


0.15 


0.15 


10.10 


9.62 


0.00 


0.00 


0.00 


6.71 


6.71 


0.15 


0.11 


0.06 


2.23 


2.22 


1.21 


0.06 


0.06 


1*.25 


1*.35 


-1.3* 


-0.06 


-0.07 


16.*6 


16.57 


-1.16 


•0*06 


-0.13 


9.12 


6.66 
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Comparison of no location eBtt-ates whan Inconsistent data 
proportions estimated for the South 



are removed (NIC) and when retained (HI) for 



RESPONSE 



I 



mc 

TU1U 

IU9C 

IU9U 

|U9G 

TU10 

TU10 

?OlO 

TU10 

TU12 

|U12 

TU12 

TU29 

TU33 

TU33 

1U33 

TU51 

iu51 

|Q52 

TUS2 

TU66 

TU66 

T066 

IU9U 

TU90 

TU90 

|U90 

TUlOl 

IU101 

IU102 

TU102 

10116 

lOllfl 

IU110 

IU129 

IW129 

|U129 

IM129 

T0131A 

IU131C 

T01310 

f U131F 

fttlSbBOX 

TU13<& 
I0136C 
IU1360 
^ HU36F 



1 

1 

1 

1 

1 

1 

1 

1 

1 

2 

3 

4 

1 

2 

3 

1 

2 

1 

2 

3 

1 

2 

1 

2 

1 

2 

3 

1 

2 

3 

4 

1 

2 

1 

2 

1 

2 

5 

1 

2 

3 

4 



!\ 

1 
1 
1 
1 
1 
1 



SAMPLE 
SIZE 

3948 \ 

3946 

3946 

3946 

3948 

3*48 

394 a 

3946 

3946 

3946 

3946 

3946 

90a 

906 
906 
3039 
3039 
1066 
1066 
1066 
3948 
3946 
1922 
1922 
1922 
1922 
1922 
1922 
1922 
1922 
1922 
. 3948 
3946 
549 
549 
3946 
3946 
3946 
3946 
3946 
3946 
3948 
1712 
1712 
1712 
1712 
1660 
1516 
1516 
1516 
1516 



Y-TRUE 

73.45 
17.20 

4.Z1 
6.07 
66.46 

30.59,. 
4.04' - 
7*26 

61.38 
14.20 
1.55 
23.07 
22.91 
7.94 
69.16 
90.75 
9.25 
18.26 
3.66 
76.09 
48.96 
51. OH 
47.56 
52.44 
18.95 
35.69 
45.36 
65.01 
4.02 
6.96 
24.00 
65.24 
14.76 
66.41 
33.59 
93.04 
1.00 
5.96 
9.65 
43.62 , 

3.54 
42.99 
73.95 
7.72 
4.71 
26.23 
18.56 
72.70 
9.69 
4,30 
21.54 



ME 
BIAS 

•2.11 
0.24 
0.11 
•0.98 
-4. A3 
-0.29 
-0.19 
-1.62 
0.24 
-0.01" 
-0.15 
-0.07 
1.57 
-0.09 
-1.48 
0.03 
-0.03 
1.21 
0.21 
-1.43 
0.31 
-0.31 
0.51 
-0.51 
• 0,63 
0.86 
-1,69 
0.24 
-0.09 
-0.07 
-0.07 
-0.02 
0.02 
-0.21 
0.29 
0.00 
0.00 
0.00 
0.03 
0.00 

o.otr 

-0.03 
0*03 
0.00 
0.0 0 

-0.10 
0.00 
0.00 
0.00 
0.00 
0,00 



ME 
RBS 

-2.87 
1.39 
2.61 

-12. 1H 
-7.03 
-0.94 
-4.70 ' 
-22.24 
' 0.39 
-0.07 
-11.11 
, -0.30 
6.65 
-1.13 
-2.14 
0.03 
-0.32 
6.62 
5.7* 
-1.83 
0.63 
-0.60 
1.07 
-0.97 
4.38 
2.40 
-3.72 
0.36 
-2.23 
-1.00 
-0.29 
-0.02 
0.13 
-0.43 
0.66 

u.oo 

0,00 
. 0.00 
0.30 
0.00 
0.00 
.0.06 
0.04 • 
0.00 
0.00 
-0.36 
0.00 

o.ob 

0.00 
0.00 
0.00 



NI 
RH» 



-2.67 
1.50, 
2.66 
-11.97 
-7.09 
-0.69 
-4.59 
-22.17 
0.26 
0.23 
-10.81 
.0.22 
6.44 
-0.99 

-2. ere 

0.02 
-0.22 
7.39 
6.53 
-2.03 
0.55 
-0.53 
1.21 
-1.10 
4.75 
3.32 
.4.60 
0.66 
.3.02 
0.28 
-1.92 
.0.06 
0.35 
0.46 
.0.91 
.0.02 
.1.77 
0.71 
.0.35 
-.0.02 
0.42 
0.06 
.0.10 
0.40 
0.40 
0.02 
0.00 
.0.17 
0.57 
0.57 
-0.25 



NIC 
RBX 

0.78 
-10.62 
-9.75 
•13.31 
-0.92 
-11.35 
-22.29 
-19.66 
2.63 
-11*76 
^-20,56 



1.44 
-9.43 
(3.19 
2.75 
0.12 ' 
,-1.26 
-21.82 
3.62 
4.93 
-0.01 
0.01 
-0.19 
0.17 
1.35 
4.18 
-4.06 
1.33 
-1.84 
-1.65 
-2.83 
-0.03 
0.19 
1.12 
-2.22 
-0.02 

«:.77 

0.71 
-0.60 
0.02 
-0.03 
0.11 
-0.06 
0.57 
0.57 
0.19 
0.00 
-0.17 
0.57 
0.57 
-0.25 



ERIC 
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NI 
BR 

-3.13 
0.26 
0,36 
-2.33 
-6.57 
-0.30 
-0.59 
-6.24 
0.29 
0.05 
-1.05 
-0.09 
1.24 
-0.09 
-1.17 
0.03 
-0.03 
0.65 
0.75 
-0.96 
0.26 
-0.26 
0.38 
-0.36 
1.13 
0.75 
-1.22 
r 0.39 
-0.19 
0.03 
-0.30 
-0.06 
0.06 
0.10 
-0.10 
-0.03 
-0.06 
0.09 
-0.06 
0.00 
0.05 
0.03 
-0.07 
0.03 
0.01 
0.00 
0.00 
-0.07 
0.05 
0.05 
-0.04 



NIC 
BR 

0.96 
-2.06 v 
-1.03 
-2.11 
-1.31 
-4.10 
-3.46 
-3.99 
2.46 
-2.66 
-1.71 
0.57 
-1.12 
0.27 
1.12 
fr*16 
-0.16 
-3.06 
0.30 
3.16 
0.00 
0.00 
-0.05 
0.05 
0.40 
0.83 
-1.09 
0.60 
-0.11 
.0.20 
-0.46 
-0.03 
0.03 
0.25 
-0.25 
-0.03 
-0.06 
0.09 
-0.10 
0.01 
0.00 
0.05 
-0.04 
0.05 
0.02 
0.03 
0.00 
-0.07 
0,05 
0.05 
-0.04 



NI 



3.01 
5.43 
6.33 
13.03 
7.13 
3.06 
6.96 
22.46 
0.94 
4.16 
14.66 

2.55 , 
10.63 
10.33 
3.52 
0.74 
7.26 
11.41 
10.84- 
2.69 
2.14 
2.06 
3.42 
3.10 
6.35 
5.52 
5.94 
2.36 
15.95 
6.78 
6.51 
0.91 
5,30 
4.61 
9.11 
0.69 
25.63 
7.89 
5.73 
2.19 
7.27 
2.10 

I. 51 
11*06 
24. Il 

5.03 
5.12 
2.37 
10.27 

II. 42 
5.57 
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NIC 
R\/fiSEX 

1.13 
11*60 
13.56 
14.72 
1*16 
11.66 
23.20 
20.26 
2.83 
12. H6 
23*61 
2.69 
12.64 
12.15 
3.68 
0.77 
7.60 
22.95 
12.26 
5.16 
2.10 
2.01 
* 3.65" 
3.31 
4.97 
6.50 
5.51 
2.56 
16.09 
6.23 
6.53 
0.90 
5.22 
4.56 
9.02 
0*69 
25.63 
7.69 
5.76 
2*20 
7.00 
2.10 

I. 5»'. 
11.1 * 
24.56 

5.07 
5.12 
2.37 
10.27 

II. 42 
5,57 



! Comparison of no Imputation estimates when inconsistent d*ta are removed (NIC) and when retained (NI) for 
proportions estimated for blacks on average ability 



tun 



RESPONSE 



O* 



IM1A I 

fttlC 1 

TU1U 1 

TQ1G 1 

TU5A 1 

?«9G 1 

?U90 1 

?«9« 1 

TU10 1 

mo 2 

fUlt 3 

mo * 

TOW * 

ma 2 

1012 3 

IU29 1 

l«29 2 

T«33 1 

T033 2 

T033 5 

T031 1 

IU51 2 

TU52 1 

T052 2 

TU6b 1 

?U66 2 

T«66 , 3 

T«90 1 

?O90 2 

?U90 3 

mo * 

TUlOl 1 

IU101 2 

IU102 1 

IV10* 2 

?*ua i 

tuna 3 

- TU129 1 

?«129 2 

?«129 3 

f«12*i * 

?«131A 1 

f fuia&c i 

TU1310 1 

ms»F i 

IU13480X I 

IU13M I 

fttlStC 1 

O r«i3*o i 

ERIC""" „ 1 



SAHPLE 

SIZE 

957 

357 

357 

357 

357 

357 , 

357 

357 

357 

357 

357 

357 

101 

101 

101 

255 

255 

107 

107 

107 

357 <* 

357 

233 

233 

233 

233 

233 

233 

233 

233 

233 

357 

357 

61 

61 
357 
357 
357 
357 

357 

357 

357 

103 

103 

103 

103 

121 

101 

v 101 
101 
101 



Y-TRUE 

66.7* 
25.26 

I. 3* 
15.60 
61.72 
12.63 

7.36 
11.23 
56.17 
13.00 

2.73 
26.03 
37.03 

7.36 
55.57 
92.75 

7.23 
32.20 

I. 26 
63.52 
30.71 
69.26 
19.0* 
50.96 
20.16 
32.16 
17.36 
37.61 

7.19 
10.63 

II. 25 
60.90 
19.10 
61.07 
36.93 
07.57 

0.23 
12.21 

6.69 
59.06 

5.21 
27.02 
71.73 

7.61 

0*. 30 
22.16 
16.65 
67.13 

II. 10 
7.21 

15.60 



\ HE 
BUS 

-1.91 
•1.25 
-0.35 
-1.26 
-7.17 
-1.56 
-0.23 
-5.26 
1.06 
0.00 
0.00 
-1,09 
-1.16 
0.00 
1.16 
0.16 
-0.16 
1.31 

o.oo 

• 1.31 
1.16 
-1.16 
0.00 
•0.51 
0.69 
0.15 
-1.11 
-0.51 
0.65 
0.76 
-0.91 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
h.00 
0.00 
0.00 

o.oii 

0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 



HE 
R8X 

-2.90 
-1.91 
-7.70 
-27. *3 
-11.51 
-3.66 
-3.11 
-16.65 
1.92 
0.00 

o.oo 

-3.66 
-3.15 
0.00 
2.06 
0.17 
-2.20 
1.06 
0.00 
-2.06 
3.77 
-1.67 
0.00 
-1.40 
3.12 
1.36 
-2.10 
-t,.35 
6.66 
7.33 
-2.12 
O.OO 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 



NX 

Hex 

-2.91 
-1.96 

• 7,65 
-27.12 
•11.53 

-3.66 
-3.09 
•16.61 
2.15 
1.13 
1.13 

• 5.55 
-1.57 

6.62 
0.11 
0.66 

-6.72 
0.71 
1.61 

-0.16 
3.65 

-1.71 

2.5a 

•2.13 
1.71 

• 0.51 
-1.66 

' 1.36 
1*.65 

• 1.99 
•3.20 
-0.26 

1.19 
0.00 

o.co 

0.35 
3.05 

•2.57 
0.79 

-0.51 
0.79 
0.79 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 



NIC 
RB« 

3.30 
•15.57 
•23.60 
-29.26 
1.61 
•11.18 
-16.27 
-15.67 
3.92 
-11.06 
5.36 
-3.21 
-23.29 
23.53 
12.39 
0.57 
-7.33 
•36.53 
20.63 
16.11 
1.22 
-0.51 
1<55 
-1.19 
3.62 
-1.23 
-0.77 
3.69 
9.29 
-10.92 
-2.09 
-0.26 
1.19 
0.00 
0.00 
0.35 
3.05 
-2*57 
1.26 
•0.69 
1.26 
1.26 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 



HI 
BR 

-0.53 
-0.36 
-0.31 
-2.21 
-2.21 
-0.56 
-J.20 
-4,20 
0.11 
0.0* 
0.02 
-0.61 
-0.07 
0.12 
0.01 
0.37 
-0.37 
0.01 
0.02 
-0.05 
0.33 
-0.33 
0.36 
•0.36 
0.25 
-0.03 
-0.26 
0.16 
0.92 
-0.10 

-r^5 

-0.06 
0,06 

o.oo 

0.00 
0.16 
0.03 

-0.17 
0.03 

-0.11 
0.03 
0.09 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 



NIC 
BR 

0.75 

•i.:5 

-1.0/ 
• 1.97 
0.27 
-2.32 
-0.93 
-3.97 
0.66 
-0.51 
0.09 
-0.32 
-1.31 
0.36 
0.96. 
0.30 
•0.30 
-3.13 
0 21 
2.76 
0.10 
-0.10 
0.20 
-0.20 
0.20 
-0.06 
-0.11 
0.13 
0.52 
-0,55 
•0.26 
-0.06 
0.06 
0.00 
0.00 
0.16 
0.03 
-0.17 
0.06 
-0.16 
0.05 
0.15 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 



HI 
R\ZHSk* 

6.16 
13.69 
v 25.33 
30.06 
12.65 

7.17 
15.69 
16.12. 

6.37 
19.61 
51.56 
10.59 
20.30 
56.71 
13.16 

I. 91 
21.91 
16.59 
72.23 

6.26 
12.23 
5.13 
6.96 
6.71 
19.25 
15.06 
6.19 
6.53 
21.61 
19.31 
7.75 
1.16 
16.93 
15.19 
23.63 
2.20 
67.01 

II. 60 
20.36 

li76 
25.21 

6.12 

6.11 
15.22 
99.00 
19.97 
21.02 

1.60 
25.32 
52.65 
22.97 



NIC 
RtfHSEft 

5.19 
20.53 
32.26 
32.61 

6.07 
. 15.76 
23.76 
•»7.09 

7.12 
23.13 
57.19 
10.35 
29.25 
66.56 
17.63 

I. 91 
21.66 
10.11 
67.91 
19.27 
11.62 

5.21 
7.66 
7.37 
19.36 
15.12 
7.00 
6.61 
20.11 
22.11 
7.70 
1.16 
16.93 
15.19 

23.es 

2.20 
67.01 

II. 60 
20.36 

1.65 
25.21 

6.67 

6.11 
15.22 
99.00 
19.97 
21.02 

1.60 
3*5.32 
52.65 
22.97 



loo 



Comparison of no imputation estimates when Inconsistent da<:a ^ removed (NIC) and when retained (NI) for<means, 
estimated for the total population 



mis 

IU16 
IU69H* 

1Q1H1M 
tUlllFB 



SAMPLE 




SIZE 


t-TRUF 


11HH5 


36,75 


11H45 




7579 


2102.30 


. 7579 


2173,09 


15089 


7039,66 


15069 


6701,41 



ME 


ME 


BIAS * 


R8» 


o.oo^ 


^ 0,00 


-0,21 


-0,13 


7,60 


0.36 


HO. 05 


l.ftH 


-17,3H 


-0.2H 


-3ft. 25 


-0.43 



0.0H 
0.1H 
1.20 
^, 36 

-0.5.7 



NIC 
RB* 

0.1H 
0,27 
0*60 
0,56 
-0*30 
-0.*1 



NI 

BR 

0.12 
0,22 
0,67 
1,50 
-0,*H 
-0,66 



. NIC 
8R 

0.H3 
O.HH 
0,H5 
0,HH 
-0,29 
-0,53 



NI 
Rv/SSE* 

0.33 
0,63 
1,61 
2,66 
1*06 
l.OH 



NIC 

0.37 
0*66 
1.45 
1.41 
1*06 
0.69 



Ju 



15:) 



C^. rlM n o f „» ^ntstion estate. *• inconsistent set. sr. removes (H,C, .ns -en tet.ined (HI, for 
estimated for males 



new 

1019 

I089HA 
IUS9H0 
IQIMFA 
tOKlFB 



SAMPLE 

size 


T-TRUC 


HE 
BUS 


nt 
RB« 


6062 
6062 

3970 
. 7357 
7357 


183.6* 
2182,70 
230*1.58 
6623.30 
82H.18 


•0.01 
•0.19 
17.98 
63.71 
-H.53 
•32.41 


•0.02 
•0.10 
0.62 
2.76 
•0.21 
•0.39 



Uf 


NIC 


ROt 


RBX 


0.04 


0.17 


0,22 


0.30 


2.10 


0.73 


3.60 


0.74 


0.02 


0.25 


0.09 


0.28 



NI 


NIC 


BR 


• 8R 


0.06 


0.36 


0.25 


0.33 


1.07 


0.46 


1.5* 


0.58 


0.01 


0.18 


0.07 


0.21 



' NI 

RtfiSEft 

0.H6 
0.92 
2.66 
4.29 
1.39 
1.27 



NIC 
RtfiSCft 

0.49 
0.98 

1.73 
i.47 
1.42 
1.32 
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CoeperUon of no Uputetion estates when Inconsistent date ere removed (NIC) end when refined. (NT) for -ens . 
estimated for individuals of high ability 



ITCH 

IQ15. 

i<*16 

IU09HA 

fQ89Htt 

IU111FA 

IU1H1F8 



SAMPLE 

sue 

2107 
2107 
21*5 
2155 
2039 
2039 



?-TRUE 
36.26 

r*8.26 

2597. 9H 
2690.50 
5327.10 
,3H9.55 



ME 


HE 


BIAS 


R8ft 


-0.01 


-0.02 


•0.10 


-0.06 


20.84 


0.95 


51.33 


1.90 


2.23 


0.01 


-22.50 


• 0.30 



HI 


NIC 


Hi 


NIC 


RB* 


RBX 


OR 


BR 


•0.00 


0.05 


-0,09 


0.05 


0.03 


0.18 


0.02 


0.1H 


0.91 


•0.17 


0.17 


-0.1H 


2.07 


0.38 


1.00 


0.25 


•0.10 


0.15 


•0.06 


0.07 


-0.H1 


•0.29 


-0.26 


-0.19 



HI 


NIC 


RvffiStx 


Rtffttft 


0.85 


0.81 


1.21 


1.81' 


2.12 


1*21 


2.93 


loM 


2.08 


2.08 


1.60 


1.56 



bt> 



• , mc > and „hen retained (HI) for «•■>» 
^ , „ mutation eaci-ces .hen inconstant data ara removed (HIC) and 



~ , -'" ut ..... .... !:t? W! : 



SSI •« .» i-u »•» >•« „•"» ...» »•»» 

ss5« b: ss-s :S: " "•" 



NI 


NIC 


BR 


BR 


0.06 


0.63 


0.16 


0.37 


1.19 


1.60 


0.** 


0.33 


-0.*3 


-0.39 


-0.73 


-0.55 



o 



b"0 
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Comparison of no imputation estimates when inconsistent data are removed (NIC) and when retained (NI) for means 
estimated for blacks 



lien 

1015 
IU16 
IQ69H* 

fttl4lFA 
tUlHlfb 



size 


t-TRUC 


ne 

BIAS 


ne 

R8X 


NI 
R8* 


NIC 
RBX 


NI 

"br 


NIC 

BR 


1H56 
1H55 
926 
926 
1963 
1963 


37.66 
1H5.73 
1702.03 
177H.33 
9945.66 
6969.57 


0.1H 
0.01 
51. H3 
30.56 
-55. 
-65.36 


0.37 
0.00 
3.02 
1.72 
-0.93 
-0.93 


0.49 
0.5H 

H. 29 

I. 9H 
-3.02 
• 1.60 


0.45 
0.69 
2.27 
-0.92 
-1.62 

-o.«u 


0.65 
0.21 
0.90 
0.31 
-1*03 
-0.61 


0.62 
0.34 
0.55 
-0.25 
-0.50 
-0.13 



NI 

R\/fi5e* 

0.90 
2t66 
* 6.42 
6.52 
4.21 
3.43 



NIC 

Q»ar» 
2.75 
4.67 
3.76 
3.61 
3.21 



1 



1 

X v> 4 



C«p.rlso„ of no taxation ssti M tas „hs« inconsistent data ara ra„ovad (NIC) and whan tst.inad (NI) fot -an. 
estimated for the South 

- u« Mir NI NIC NI NIC 

»« :-2 ;-a 1:S *!:2 S3 ! : » 

KSS SS S5:S ■ : .: ...ti 
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Comp^ison of no imputation estimates when inconsistent data are removed (NIC) and when retained (NI) for means 
estimated for blacks of average ability 









HE 


HE 


tun 


size 


Y-TRUE 


BIAS 


RBX 


1015 


256 


37.11 


0,06 


0.16 




256 


162.69 


-1.02 


-0.62 


IU69HA 


233 


1690. H6 


-12.H9 


-0.66 


roa9Hb 


233 


197H.H3 


-19.62 


-0.99 


I0141M 


357 


56H6.06 


39. i*6 


0.69 


IU1H1FB 


357 


6901.66 


•2.92 


-0.0<* 



NI 


NIC 


NI 


NIC 


NI 


NIC 


RBX 


RBX 


BR 


BR 


RvffiSEX 


fttfH§Eft 


0.66 


0.50 


0.33 


0.25 


2.06 


2.0/ 


0.9H 


1.61 


0.11 


0.19 


7.95 


6.26 


-2,03 


1.45 


-0.32 


0.20 


6.51 


7.22 


• 0.27 


0.92 


-0.03 


0.11 


7.26 


7.63 


3.26 


H.l* 


0.H5 


0.55 


7.90 


6.51 


-0.H3 


1.2S 


•0.06 


0.17 


7.01 


7.19 



I 

I 



r 



9 
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Appendix G 



COMPARISON OF NO IMPUTATION ESTIMATES WHEN 
INCONSISTENT DATA ARE REMOVED (NIC) AND WHEN RETAINED (HI) 
FOR SELECTED CROSS-TABULATIONS 



73 
♦ o 
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Glossary of Terms Used in the Tables 

i 

SAMPLE SIZE - number of sample members eligible to respond to a particular 
item for the domain under consideration. 

Y-TRUE - the estimate obtained using the telephone corrected and com- 

pleted data. 

NI - estimates obtained using do imputation or editing procedure on 

the experimental data set. 

I . MIC - estimates obtained using no imputation or editing procedure on 

\ the experimental data set after inconsistent data are removed. 

ME - • measurement error caused by the use of data containing logical 

inconsistencies . 

RB% - the_ relative bias defined to be the bias divided by the value 

of Y-TRUE, expressed as a percentage. 

BR - the bias ratio defined as the bias divided by the standard 

error of the estimate. 

RVMSE% - the relative root mean square error defined as the square root 

of the mean square error divided by the value of Y-TRUE and 
expressed as a percentage. 
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^ - no — - *- — - - - — (HIC> ^ "" ln ' d <NI> 

Ii£"«bul<>tlo.» ot the tot.l population 



MKWS-TAIUM.ATION 



TQ1A * 
TlftA x 
TQ9A * 
*|9A x 
Tf|13tA 

t»iniA 



TQIC 

Hi*: 

x TQI36C 
x TT)1W 



SAfiPLE 
SIZE 

15069 
15009 
15009 
15009 
6336 
6336 
57% 3 
57*3 



Y-IRUE 

10.21 
2*97 

17.21 
2.60 
5.11 

10.12 
5.06 
7.« 





fit 


BIAS 


Rttft 


-1.12 


•10.97 


0.15 


5.01 


-•1.25 


-21.65 


-0.33 


-12.32 


0.00 


0.00 


0.02 


0.19 


0.00 


0.00 


0.01 


0.13 




NIC 
HBX 

-9.01 
-7.50 
-20.62 
-20.05 
0.52 
0.13 
0.39 
-0.63 



NI 

BK 

•3.30 
1.03 
-1H.76 

-2.71 
0.03 
0.10 
0.01 

-0.09 



NIC 
BR 

-2.61 

-1.36 
-13.02 

-5.31 
0.05 
0.09 
0.01 

-0.13 



NI 

11.29 
6.55 
21.76 
12.76 
6.00 
1.37 
0.01 
1.06 



NIC 
ftVnSCtt 

9.63 
9.31 
20.67 
20.10 
0.06 
1.36 
0.05 
1.90 
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'Comparison of no imputation estimates when inconsistent da 
cross-tabulations of males 





SAMPLE 




ME 


tic 


CKOSS-TAflM.ATlOh 


sue 


Y-TRUf 


BIAS 


H8V 


T<)tA » TQIC 


7367 


11,69 


-1.53 


-13,00 


11)1 A x Tl)U> 


7357 


3,6% 


0.20 


5.H9 


TQ9A x TI)9C 


7357. 


1ft, 05 


-H.'.l 


-22,82 


WIA x T<)9l> 


7357 


3,46 


-0.36 


-10. HO 


v}t ha x unite 


2592 


3,69 


0.00 


0,00 


V)i HA x TQUtF 


2592 


22 # 90 


0,0*4 


0.17 


TOI I6A x TQI16C 


2231 


3,7ft 


0,00 


0,00 


Ti)ll6A x TQI16F 


2231 


17.66 


0,00 


0,00 



I 




ERLC 



are retrieved (MIC) and when retained (NI) for 



NI 


NIC 


NI 


HU% 


RB* 


BH 


-12,96 


-9.66 


-2.93 


5,51 


•5.31 


0.99 


-23,13 


-20.27 


, -fi.Hft 


-10,05 


-16*97 


-2.26 


0,25 


0.5ft 


0,02 


0,30 


O.Hft 


0,12 


0,62 


0,65 


0,04 


• 0,50 


-0.H7 


-0,10 



ni r 


NI 


Nlr 


BR 


¥ not ^ 


nvn>)La 


-2.15 


13.69 


10.66 


-0.96 


7.60 


7.65 


-9.56 


23.29 


20.36 


-3.07 


10.96 


17. 6H 


0.05 


11.00 


11.12 


0.19 


2.H5 


2.49 


0*05 


12.92 


12.93 


-0.10 


H'.6H 


1.63 



~.<>A fNtn and when retained (NI) for 
Co-o.tison of ... i-put.tion estates -hen inconsistent date ate te«oved (NIC) 
c^tXl.tlons 7or individuals of high ability 



<:WISS-TAM1II.AT10N 

TQiA « TQIC 
Tf|IA * TQID 
TQ9A * TIJ9C 
TQ9A x TQ9D 
TQl )IA * TQHIC 
TIjntA K *|I1IF 
T9I36A k TQ136C 
T«)IV»A x TQI36F 



SAMPLE 

sue 

2039 
2659 
2659 
2639 
6b7 
667 
*31 
731 

/ 



Y-TRUk 

16.36 
2.73 
32.15 

l.« 

9.52 
12.61 
10.67 

6,01 



me 

BIAS 

•1.66 
0.02 
-6.H3 
-0.24 
0.00 
0.00 
0.00 
0.00 



ME 
ROft 

>*U.27 
0.73 
•26.22 
~12.*6 
0.00 
0.00 
0.00 
0.00 



NI 
BBS 

•10.10 
0.66 
•26.29 
-12.00 
0.21 
0.21 
0.16 
0.16 



NIC 
RHX 

-6.27 
-5.24 
•16.47 
-7.65 
0.21 
0.21 
0.16 
0.16 



NI 


NIC 


NI 


NIC 


t)R 


BR 


RVMSEV 


RVMSEX 


•2.25 


-l.*9 


11.06 


7.54 


0.04 


-0.40 


13.61 


14.01 


12.74 


•6. '43 


26.37 


16.67 


-0.66 


-0.49 


16.36 


17.65 


0.01 


0.01 


13.34 


13.34 


0.01 


0.01 


15.30 


15.30 


0.01 


0.01 


13.94 


13.94 


0.00 


0.00 


17.70 


17.70 



-J 
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I 
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Comparison of no imputation estimates when inconsistent data are removed (NIC) and when retained (HI) for 
cross- tabulations for individuals of low socio-economic status 



(mSS-TABUUTtON 

Ti>l A x TQIC: 
TI)tA x T4)U> 
Tl)9A n TQ9C 
T09A x TI)9I> 
fi|| MA x TIJI )l<* 
I'll IIA x iglllF 
|t}| Jf>A x V)\ 36*: 
Ti)l I6A x TljnftF 



SAMPLE 




tit 


tie 


NI 


NIC 


NI 


NIC 


Ni 


NIC 


SHE 


t-TBUC 


BIAS 


8BS 


ROft 


ROX 


BR 


BR 


RVftS£% 


RVftSEft 


H220 


5.9a 


-o.a7 


• m.55 


•1H.H0 


-16.02 


-1,98 


-1,92 


16.13 


18.05 


H220 


2.H9 




17.70 


17.90 


-10.04 


1,87 


-1.06 


20.29 


13.78 


H220 


9.26 


•2.35 * 


-25.33 


-2H.96 


-2H.77 


-*.93 


-*.an 


25. H7 


25.29 


H220 


1.98 


0.06 


3.03 


3.68 


-16.21 


0.51 


-2,20 


a. 07 


19*99 


2093 


3.15 


0.00 


0.00 


0,35 


0.7H 


0.02 


0.05 


13.69 


13.60 


2093 


a. 7? 


0.09 


1.03 


1.02 


0.37 


0.11 


0*05 


7.18 


7.19 


2021 


3.61 


0.00 


0.00 


0.15 


0.23 


0.01 


0*02 


10.60 


10.59 


2021 


6.ia 


0.05 


o«ao 


0.8S 


0.23 


0.10 


0.02 


8.29 


8.39 
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Comparison of no imputation estimates when inconsistent data are removed (NIC) 
• crosa-tabulatioos for blacks 

SAMPLE "E HE NI 



CfKBS-TABM-ATION SUE Y-TRUt 



BIAS RBX BBS RBX 



T,,i» . TMC ' 1963 9.71 -12.31 r»2.12 -12.01 

»A?»D 1»« 2W -0.19 -6.31 -6.16 -17.16 

^m"S!c 1963 15 0- -3.90 -23.92 -25.39 -20.65 

32£"JS 1963 3 37 iO.15 -12.60 -11.09 -17.33 

J, r;,3.C "m 3 99 0.00 0.00 0.99 1.99 

S Sllf 611 9.S5 0.16 1.62 J.JJ 0.70 

s J:S S:S S:S !:S 1:" 



when 


retained 


(NI) for 
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COMPARISON OF HOT DECK AND WEIGHTING CLASS 
ESTIMATES WITH NO IMPUTATION ESTIMATES 
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Glossary of Terms Used in the Tables 

SAMPLE SIZE - number of sample members eligible to respond to a particular 
item for the domain under consideration. 

Y-TRUE - the estimate obtained using the telephone corrected and com- 

pleted data. 

no imputation-iefers to the estimates obtained using no 
imputation or editing procedure on the experimental data set. 

hot deck-refers to the estimates obtained after the experimen- 
tal data set had missing data and violations of routing pat- 
terns corrected using a hot deck procedure. 

weighting class-refers to the estimates obtained after the 
experimental data set had missing data and violations of 
routing patterns corrected using a weighting class procedure. 

the relative bias-defined to be the ratio of the bias to 
Y-TRUE, expressed as a percentage. 

the bias ratio or the bias divided by the standard error of the 
estimate . 

RJSSE% - • the relative root mean square error-defined to be the square 
V root of the mean square error divided by the value of Y-TRUE, 

expressed as a percentage. 
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0,00 


0,01 



Rl/HSEX 

3.33 

3.21 
13.17 
11,24 
6,46 
2,63 
4,09 
20,56 
0,72 
9.35 
13.54 
2.59 
4,7? 
6 f 56 
3.3? 

0. 36 
3.36 
9.07 

16,86 
3.00 

1. ^7 
1.54 
2.16 

l.« 
3.23 
4,65 
4.51 

I.I* 
14,36 

9,72 

3,63 

0.77 

4.36 

3,64 

5.77 

0,42 

12,61 
3,97 
5,25 
1.70 
6,6? 
2,02 
1,64 
6,66 

23,40 

3,56 
2,00 
7,44 

ie.75 

2.26 



r/mSIx 

3.35 
3.14 

13.23 

11.07 
6,46 
2,60 
3,83 

20,76 
0.74 
4.37 

13.66 
2,66 
4,54 

10,46 
;,44 
0.36 
3.42 
7.29 

12.56 
2,36 
2.12 
1.65 

2. if 
1.60 
2.53 
4.90 
3,66 
1,84 

14,31 

10,24 
4,14 
0,60 
4,49 
3,96 
6,30 
0,45 

13,09 
4,04 
5,26 
1,70 
6,65 
2,01 
1.67 
8,70 

25,14 
1.53 
3.5? 
2.15 
7.17 

16,69 
2,34 



Nt 

hVhSIx 

3.4/ 
3,04 
13.01 

il.ii 
6,51 
2,44 
4.7-i 

<ro,/9 
0./6 

4,41 
13,66 
"2.46 

4,20 ' 
13,16 

3.21 
0,74 
6,94 
7,06 

12. 1? 

*2,i8 
2,04 
1.5V 

2,!4 
3.03 
4,96 
3, 60 
1.63 
16.42 
10.94 
3,5? 
0.64 
4.71 
3,6i 
6,06 

0. 54 
16,25 

4,22 
5,29 

1. /i 

6,60 
2,01 
1.59 
10,49 

22, ?5 
1 ,50 
4,04 
1.96 
8,92 

20,86 
2.26 



ERLC 



193 



191 



7 



O-WI h*** ano «\ g htin 8 cuss estates with no l-potnclpn estates of proton, for lnol.lou.ls 
of high ability 



Cn 
l 



/ 




IIEH 

T01A 
TOlC 

rota 

TQ1C 
?Q9A 
TQ9C 
T090 
TQ9G 
TQ10 
TQ10 
TQ10 
fQlO 
T012 
TQ12 
IQ12 
TQ29 
TQ29 
TG55 
TQ51 
TG55 
tQ5t 
TU51 
fQ52 

|U52 

T066 

f(J66 

T066 

fO^o 

TQ90 

TQ90 

T(l90 

TQ1 01 

foioi 

TOI02 
fO!02 

toil* 

TQl 16 
T01 1» 
TU12* 
TQ12* 
TU12* 
fQt2* 

rcinu 
ionic 

T0U10 

T01SIF 

I0I566OX 

TQ156A 

T0U6C 

IU156D 



RESPONSE 



SAMPLE 
SUE 

2859 
285* 
2659 
21159 
2659 
2659 
2659 
2659 
2659 

265* 
2659 

2659 
752 
752 
752 
2107 
2107 
995 
99J 
995 
2659 
2859 
2155 
2155 
2155 
2155 
2155 
2155 
2155 
2155 
2155 
2659 
285* 
951 
9S1 
2659 

263* 
2659 

265* 
2659 
265* 
2659 

667 
667 
667 
667 
900 
751 
71! 
751 
751 



/ 



Y-TRUE 

71,74 
26,26 
4,11 
6,56 
65,08 
58,7* 
2,66 

4,*1 
55,65 
20,40 

I. 26 
24,70 
22.05 

6,26 
71.72 
69,62 
10.16 
9,59 
2.55 
66,26 
21,90 
76.10 
47,56 
52.62 

II. 65 
52.40 
55,95 
62.00 

5.15 
5.26 
27,59 
60,52 
55,66 
60.71 
59,29 
9)1,05 

O.H 
5.17 
^-*4,75 



56,00 
2.57 
26,66 
76.06 
12.65 
0,61 
22.90 
22,20 
72,05 
17,65 
0,05 
17,55 



Nl 
RBX 

•0,25 
0,05 
1.55 
-11.42 
•10,60 
-0,75. 
0.5* 
-55,59 
-0.50 
-0,52 
-2.55 

1.2» 
0,96 
1.47 

-1.65 

•0.O5 
0.52 

17.65 
7.90 

•2.11 
1.4* 

-0.41 

0.06 
-2 ,'60 
5.77 
-1.60 
-0.07 
2.4* 
•0.16 
-0,2k 
0t0b 
-0,0* 
0.25 
-0,5* 
-0.02 
-15.67 
1.55 
0,05 
•0.06 
0.55 
0.U 
•0.0'i 
0.21 
a, 21 
0.21 
0.00 
•0.06 
0,16 
0.16 
0.16 




HD 
RBX 

.0,1/ 

0,24 

11.5* 
14. 6i 
-0,70 
0,20 
55, 5i 
-0,54 
0,61 
-2,62 
0,7/ • 
0,60 
1,65 
-0,55 

0,49 
02 
0,00 

-1,1? 

I.3l 
•0,56 
0.50 
•0.5i 
-5,52 
5,49 
• 1.3« 
•0,0? 
0,0/ 

-i.ot 

0,50 

o.i* 

.0,24 
0,20 
-0,5B 
-0,02 
-H,96 
1.6/ 
-0,2* 

o.oi 

0.00 
0.05 

o,oi 

-0.05 

'-o.oi 
•o.oi 

.0.54 
0,0$ 
• 0.1* 
-0.13 
-O.lJ 



MC 
RBX 

-0,56 
0.02 

1.3* 
•11.50 

•14.57 
•0.60 
5,61 
-55,51 
-0,55 
0,52 
•2,62 

0, *0 

1. *l 
6,05 

•1.26 
•0.56 
• 5,17 
11.0* 

*.71 
-1,46 

1.70 
-0.47 
-0,1* 

0.17 
-2.07 

5,25 
-1.35 
•0,04 

7,66 

-l.ol 

-0,16 
•0.02 
0,04 
0,55 
•0,65 
•0,05 
-5,05 
0,61 
0,1* 
-0,0* 
0,00 
0,11 
•0,58 
1.66 
6,97 
0,21 
-0,10 
-0,56 
1,56 
7,26 
-0,15 



NI 
BR 

-5,01 
0,00 
0,15 

-2,02 
•10,55 

•0,59 
0,02 

• 5,50 

•0,15 
•0,16 
•0,11 
0,27 
0,60 
0,06 
•0,60 

• 0,00 
0,00 

1.2* 

0,55 
-1,42 

0,56 
•0,56 
-0,0$ 

0.03 
•0,45 

;,04 
•0\90 
•0.04 
. 0,15 

• 0,01 
•0,05 

0.02 
.0,02 

0,14 
•0.14 
•0,0* 

• 0,60 
0,20 
0,00 

•0,04 
0,02 
0,03 

• 0,02 
0,02 
0,00 
0,01 
0,00 

•0,01 
0,01 
0,00 
A. 01 



M0 
BR 

•5,00 
0,06 
0,14 

•2,05 
-10,57 

• 0,50 
0,01 

• 5,50 

• 0,20 
0,20 

-0,15 
0,17 
0,07 
0,06 
-0,10 
-0,07 
0,07 
0,76 
0,00 

• 0,77 
0,55 

-0,55 
0,11 
•0,11 

• 0,50 
0,96 

• 0,60 

• 0,51 
0,25 

•0,10 
0,10 
0,06 

• 0 , " ' 

o. 

•0,.- 

• 0,06 

o,r* 



06 
15 
15 



w 66 
0,22 
0,00 



o. 



,02 
00 



0,01 
0,00 
0,00 
0,00 
0,00 

•0,00 
0,01 

•0,01 
0,00 

-0,01 



MC 
BR 

• 5,00 

0.11 
0,14 

• 2,05 
•10,50 

•0,52 
0.25 
•5,62 

• 0,25 
0.17 

• 0,15 
0,20 
0,25 
0,46 

• 0,45 

• 0,05 
0,05 
0,60 
0,56 

-0,96 
0,41 
•0,01 

• 0,06 
0,06 

• 0,56 
0,60 

•0,65 
•0,20 
0,46 
•0,12 

•o,co 

•0,01 
0,01 
. 0,26 
•0,26 
•0,05 
•0,06 
0,06 
0,02 
•0,05 
0,00 
0,05 
-0,17 
0,16 
0,2* 
0,01 
•0,01 
•0,10 
0,15 
0,26 
•0,01 



R^MSEX 

3.4* 
9 f 92 
12,74 
14,67 
2,01 
13,60 
54,72^ 

2,80 
20,94 
4.56 
9,10 
17, 2? 

3.1? 
0,77 
6,66 
22,57 
16,60 
2.57 
0,54 

1,21 

I, 98 
6,50 
5.21 
2,00 

»••* 
16,60 

12,50 

0,62' 

2,05 

5,71 

I. 7? 
2,76 
0,02 

26,50 
7.56 
6,75 
1.7* 

II. SJ 
5,26 

2.U 
10,52 
22,50 
11,15 
6.99 
5,00 

*.M 

26,65 

II. 54 



HO WC 
R/S3EX R/fiStX 



1 



0,5* 

3.44 

9,66 
12,64 
14,6* 

2.0? 
13,56 
30,6b 

5, OS 
20,6% 

O t 05 

6,26 
20,42 

5.17 

0. 76 
6,74 

17,61 
13,50 

1. *t 
17 
17 

-.** 
2,6* 

7.37 

0. 96 
2,55 

I. 5? 
16,10 

9,70 
4.67 
2,04 
5.66 

1. *l 

2. *5 
0,02 

26, *3 
7,66 
6,47 
1.75 

11,60 
3.23 

2.U 
10.55 

22.06 

II. 04 
6,8* 
5,55 
9,07 

26,73 
11. 17 



4,59 
5.bf 
9,86 
12,64 
i4,64 

14,5? 
34,76 

5,00 
20, 6S 
.51 
,68 
18.41 



4, 

7, 



11 
,69 
VI 



17,60 
iv,l« 

2,15 
o.oO 
1.23 
5,03 
. 2.73 
7.3i 

2,5ft 

I. 65 
17,55 
il.l* 

4,01 
2,0? 
3,/3 
2, OS 
3.17 
0.56 
45,68 
'9,20 
6,60 
1.7* 
12. 2C 
3*30 

2,1* 
10,1* 

io,7* 

I I, 2* 
6,56 
3.47 

10,24 
26,26 
1 1 .2<3 



-Ifto 



of proportions for 



00 
I 



I TEH 

rou 

T01C 

roio 
ruic 

TU9A 
T09C 

roio 

T010 

roio 
roio 
rui2 
roi2 
rou 

rQ29 
T029 

ru35 
ro33 
ro33 
rosi 
rosi 
ro52 

rQ52 

ro66 
ro6o 

r066 
TU90 , 
1090 

ro9o 
rove 
roiot 
roioi 

.rqto2 

TQ102 ' 

rone 
rone 
roTia 

T0129 
TQ129 

roi29 

roi29 

roniA 

rui3ic 

fqmid 

roniF 

roi36aox 

TQ136A 
T0136C 
TO1360 

T01361F 



RESPONSE 

1 
J 
1 
1 
1 
t 
1 
1 
1 
2 
3 

a 

2 

3 

1 

2 

1 

2 

3 

1 

2 

1 

2 

t 

2 
3 

1 . 

2* 

3 

4 

1 

? 

1 

? 

1 

2 

3 

1 

3 

1 
1 
1 
1 
1 
1 
1 
1 
1 



SAMPLE 
SUE 

(1220 
4220 
4220 
4220 
4220 
4220 
4220 
^4220 
' 4220 
4220 
4220 
4220 
1069 
1069 
1069 
3150 
3150 
M099 
1099 
1099 
4220 
4220 
1424) 
1420 
1420 
1420 
1420 ■ 
1420 
1420 
1420 
1420 
4220 
4220 
230 
230 
4220 
• 4220 
4220 
'4220 
4220 
4220 
4220 
2093 
2093 
2093 
2093 
235« 
2021 
2021 
202t 



Y-rRUl 

69,91 
10,77 
3,84 
11,47 
66,65. 
16,26 
3,66 
10.13 
62,88 
•9,10 
1.93 
26,09 
30,32 
7,30 
-62,39 
91, 4* 
6,54 
26,92 
3,71 
69,36 
64,83 
35,17 
49,76 
.50,22 
30,72 
29,91 
39,37 
52,38 
6,44 
6,32 
32,66 
94,03 
5.97 
67, iZ 
32,68 
90,23 

6,33 
V9,35 
*35,96 

6,12 
46,57 
71,01 

5,12 

3,97 
26,91 
14,14 
71,61 

5,72 

3.65 
21.46 





HO 


pax 






mi .51 


•0.50 


• 0, 36 


1 1,04 


1 0 .9 £ 


•12, 7 7 




•3*46 


• 3 , 46 


• 4,46 


• 3,6? 


6,66 


7 . 15 ' 


-19,92 


•20 , 26 


•0,1 3 


•0,38* 


1,03 


2.36 


a, 17 


2. 92 


• 0,35 




4.32 


f . 46 


•0,73 


• 0 , 41 


"2,01 


• 0 .66 


. 0,00 


A 1 £ 


0,09 




3. 7 i 


1 86 


10,93 


1,3* 


•2,02 


.0 79 


0,04 ' 


o ?8 


• 0,09 


-0,52 


Ii20 


• 1,7b 


•1,19. 




2.34 


•2,26 


3,*3 


9,82 


•4,43 


.5,69 


•0,01 


0,4$ 


6,94 


9 t 8V 


2.52 


•5,06 



-1.97 
-0.C2 
0,36 
•1,02 
2,10 
s -0,14 
3.51 
- 0.94 
-0,37 
•0.17 
*0.29 
"^-0|24 
0,19 
-1.34 
-0,38 
0,10 
. 0,00 
• 0,06 
0,15 
•1,73 
0,35 



9 

ERLC 



• 1,44 
0,06 

• 0,97 
0,96 

•2,00 
•0,02 
rt ,0U 
0,30 
0,44 
0,0f 
•0,5/ 

• 0,07 
0.26 

• 1,62 
•0,20 

0,05 
0,10 
•0,16 
0,14 

• 1,74 
0,2* 



HC 
RBX 

- -1.54 
0,41 
10,95 
-12,83 
-3,67 
•3, 08 
9.37 
•19,92 
•0,38 
5,10 
*.92 
-1,07 
li85 
3,96 
•1,36 
•0,51 
5,51 
\2.29 
7,64 
-1,29 
•0,13 
0,25 
•2,04 
2,03 
•2.9*" 

9,16 
•4,62 
•0,1 a 
18,28 
•6,65 
•1 ,61 
•0,24 
3,67 
2,64 
•5,40 
-0,25 
6,68 

1.22 
-0,16 

0,32 
-0,77 
•0,11 
-0,17 

5,07 

6,00 

0,63 

1.23 
-0,44 

1 ,65 

4,67 

0,57 



-2. 
•2. 

-o,< 

0,< 
-0,( 

o, 

o.. 

•0,1 



NI 

BR 

-1|24 
•0,07 
1.29 
?.92 
J. 59 
.71 
,69 
i 7 5 
.09 

.36 
.11 
1.22 

• 0,06 
-1,01 
•0,01 

0,01 
1,06 
0,96 
-1,37 
0,03 

• 0,03 
0,3i 

• 0,34 
0,46 
0,71 

•1,09 
0,00 
0,57 
0,40 

• 0,33 

• 0,04 
OfcOt 

•0,H 
0,14 

•0,23 
0,16 
0,13 

• 0,05 
•0,08 
•0,03 

0,1$ 

0,09 
•0,06 
•0,03 

0,02 

0,00 

• 0,03 
0,01 

•0,lf 
0.06 



HO v 
BR 

•1,26 
•0,05 

1.24 
•2,93 
•2,55 
-0,61 
0,94 
-2,81 
•0,26 
0,46 
0,25 
•0,03 
0,43 
•0,03 
•0,33 
.0,27 
•0,27 
0,59 
0,08 
•0,48 
0,17 
•0,17 
•0,45 
0,45 
• ^0,46 
1,60 
•1,44 
0,10 
0,72 
•0,54 
•0,26 
0,10 
•0,10 
0,13 
•0,13 
•0,04 
6,00 
0,04 
0,06 
0,03 
-0,06 
-0,04 
0,13 
-0,10 
•0,01 
0,01 

. o 0 oi 

•0,11 

0,01 
•0,17 

0.05 



19" 



HC 
BR 

•1,34 
0,01 
1,28 
•2,93 
•2,69 
•0,46 
1,18 
•2,79 

•0,29 
1,02 
0,25 

•0,34 
0,56 
0,30 

•0,70 

•0,56 
0,58 
0,64 
0,'37 

•0,78 

•0,10 
0,10 

•0,4* 
0,46 

• 0,64 
1,86 

•1,17 
•0,02 

1,16 
•0,46 
•0,30 
•0,46 

0,46 

0,38 

• 0,38 
•0,33 

0,38 < 

0,15 
•0,02 

0,111 
•0,06 
•0,07 

• 0,06 
0,27 
0,55 
0,12 
0,19 

•0,29 
0,15 
0.41 
0,10 



NI 

1,69 
6,68 
13,95 
13,49 

*.7i 

7.70 

io 9 oi 

21,20 

4,49 
12,31 
3,o9 
5,58 
11,39 
2,8,3 
0,54 
5,81 
. 5,08 

15, ?i . 

2,50 
1.57 
2,90 
3,81 
3,77< 
5,36 
5,89 
5,99 
4,41 

l3,M 
6.77 
*.17 
0,56 
8,9<| 
7,36 

15, oi 
0,64 

19,74 
7.2* 
7. Si 
2,19 
9,0$ 
1.56 
2,05 

15,50 

11.7* , 
5,07 
6 99 
1.S9 , 

10,14 

lo.ii 

5.21 



nO 

1.93 
. 6,66 
14,05 
14,56 
i.li 
7,40 
10,45 
21,51 
1,50 
5.43 
11,96 
3,16 
3,70 
12, 9j 
2,05 
0,61 
6,62 
3,66 
14,85 
1,60 
1,69 

i.li 

4,19 
5.34 
11,24 
6,9* 
".77 
16.85 
»0,61 
5/24 
0,57 
9,02 
7,40 
15,12 
0,60 

ltr,9o 

7.37 
7,43 
2,31 
9,08 
1.53 
2.14 
15,61 
11,66 
5.22 
7.31 
• 1,.60 
10,49 
10, 15 
5,32 

19 c i 



HC 

l.»i 

6,75 
Ii,«9 
iJ,i6 
i.ttS 
7,0$ 
12. 2i 
il.lt>- 
A, si, 

11,82 
J, 26 
J, 77 
13, 7B 
2,i6 
.02 
.V* 
.24 
21, iV 
2.10 
iB 
55 
b« 
65 
Si 
40 

Oi ' 

o5 . 
94 



I. 
10, 



1 

2. 

<l 

1. 
5, 
10, 

6. 

5, 
21 
16,(10 
"5.1.5 



5* 
.25 
.12 
.14 



8 



.93 
.95 



0, 

9 

7 
15 

0 
24 

7 

7 

2,26 
9,47 
1,49 

2.1* 
19,44 
16,44 
5,20 
6,60 
1.57 
10,69 
i2,lfl 
'$•34 




Comparison of hot d 



Jck and weighting class estimates with no imputation estimates of proportions for blacks 



en 
I 



1C1Q 







• SAMPLE 


ITCH 


RESPONSE 


SIZF 


TQU 


1 . 


1963 


TQIC 


1 


. 1963 


TOID 


1 


1963 > 


TOIG 


1 


1963 


T09A 


I 


1963 


TQ9C 


1 


1963 


T09D 


1 


1963 


T09C 


1 


1963 


?UlO 


1 


1963 


roio 


2 


1963 




3 


1963 


TOlO 


i 


1963 


TO! a 


1 


SOS 


TQ12 


2 


505 


TOU 


-3 


505 


T029 
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Conparison of hot deck aod weighting class estates with no Mutation estates of proportions for blacks of 
average ability 
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Comparison of hot deck and weighting class estimates with no imputation estimates of means for individuals o. low 
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Comparison of hot deck and weighting class estimates with no imputation estimates of means for blacks of average 
ability 











in 




WC 






HI 




HI) 




NC 


NJ 




HO 


WC 




ITt" 




?-TRUF 




RBI 


RB% 




8R 




8R 




bR 


r/msf.x 


R^HSEX 


r/RSIi 


TOlb 


<!S6 


17. 


11 


0.6o 


0.10 


0| 


79 


0 


.** 


0 


.11 


0 


• 17 


2. 


Ott 


2 


.25 


2. 


25 


T9!6 




16«> # 


69 


0.9<i 


-0.16 


0, 


«9 


0 


• it 


-0 




0 


,06 


7. 


9S 


7 


,6& 


7. 


96 




211 




Mfl 


-2.0i 


1 .66 


1. 


62 


• 0 


.*2 


0 


,26 


0 


.69 




51 


6 


,52 


<>. 


71 


?:J89MR 


2*1 


197<| # 


<JJ 


-o # ;.7 


1.03 


«. 


<i7 


-0 


,01 


0 


.t2 


0 


.71 


7. 


26 


a 


.17 


7. 


67 




557 


S6«d, 


06 


J. •* 


1.77 


*. 


61 


0 


• «5 


0 




6 


.51 


7. 


90 


7 


,60 


7. 


6/ 


Tomifu 


1S7 


6901. 


66 




0.60 


•o, 


01 


-0 


.06 


0 


,oa 


0 


,00 


7. 


01 


7 


,27 




55 



I 

ON 
OO 
I 



1 



i 



Appendix I 

COMPARISON OF HOT DEtfC AND WEIGHTING CLASS 
ESTIMATES FOR ALL DQHAINS FOR SELECTED ITEMS 
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Glossary of Terms Used in the Tables 

SAMPLE SIZE - number of sample members eligible to respond to a particular 
SAMPLE SIZE ^ ^ c0Qsideratl0n . 

the estimate obtained using the telephone corrected and com- 
pleted data- 

no imputation-refers to the estimates obtained using no impu- 
tation or editing procedure on the experimental data set. 

u n * j.rk—refen to the estimates obtained after the experimen- 
m ' h t °ai data set had hissing data -and violations of rout.ng pat- 

terns corrected, using a hot deck procedure. 

■i _i *«,..«• f„ the estimates obtained after the 

ing patterns corrected using a weiguuj.u 6 

the relative bias-defined to be the ratio of the bias to 
Y-TRUE, expressed as a percentage. 

the bias ratio, defined to be the bias divided by the standard 
deviation of the estimate. 

expressed as a percentage. 
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Comparison of hot deck and weighting class estimates with no imputation estimates for TQ12, response 1 
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Glossary of Terms Used in the Tables 



SAMPLE SIZE 

Y-TRUE - 

„STDERR SD - 

BRR SD - 

SD RATIO - 
NI - 

NIC - 



number of sample members eligible to respond to a particular 
'item for the domain under consideration. 

the estimate obtained using the telephone corrected and com- 
pleted data. 

the standard deviation obtained when the STDERR Taylor Series 
linearization is used. 

the standard deviation obtained using the Balanced Repeated 
Replication Technique. 

the ratio of the STDERR SD to the BRR SD. 

the no imputation estimates obtained using the experimental 
data set. 

the no imputation estimates obtained using the experimental 
data set after inconsistent data were removed. 



9 

ERIC 



-185- 
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1 
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TOtC 
TQIO 
TOlC 
T«9A 
T4I9C 
T«9D 
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1Q10 
TQl* 
TQIO 
* TQ10 

1012 

T012 

T029 

T029 

TQ35 

T033 
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TOM 

1051 
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T0101 

toioi 
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101 16 

10129 
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101310 
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H 
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sizt 

1963 

1963 
1963 
1963 
1963 
1J63 
1963 
1963 
1963 
1963 
1963 
1963 
509 
503 
303 
1*33 
1*35 
369 
569 
569 
1963 
1963 
926 
9?» 
926 
926 
926 
926 
926 
92ft 
926 
1963 
1963 
160 
lftO 
1963 
1963 
1963 
1963 
1963* 
1963 
1963 
611 
til 
611 
£11 
715 
ftlS 
613 
613 
613 



?-IRUC * 

69.37 
17.03 

3.51 
11.15 
64.93 
26.71 

6.63 
13.61 
59.55 

12.36 

I. 65 
26.21 
36.67 
12.15 
16.66 
92.03 

7.97 
37.65 

S.ttO 
56*35 
50.21 
19*79 
M.91 
55.06 
23.06 
35.77 

II. 13 
12.92 

6.37 
10.97 
39.75 
90.16 
9.61 
61.15 
35.65 
69.01 
1.21 
9.72 
12.11 
31.15 
1.21 
32.50 
73.51 
6.67 
1.02 
^^.12 
17.27 
73.11 
ft. 54 
1.76 
16.05 



NI 

STOCKS SO 

1.33 

1.12 

0.56 

1.07 

1.35 

1.22 

0.73 

0*60 

1.36 

0.92 

0.11 

1.31 

3.13 

1.79 

3.01 

0.61 

0.61 

2.12 

1.52 

2ol9 

1.31 

1.51 

2.00 

2.00 

1.66 

1.69 

1.63 

2.11 

1.17 

1.23 

2.05 

0.61 

0.61 

3.99 

3.99 

1.06 

0.31 

1.01 

0.90 

1.17 

0.51 

1.53 

2.32 

1.27 

0.96 

2.17 

1.76 

1.96 

1.22 

1.03 

1.68 
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0.67 
1.16 
1.35 
1.7ft 
0.69 

1.05 

1.21 

0.71 

0.33 

1.10 

2.77 

2.00 

2.63 

o.oi 
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1.90 

1.17 

1.72 

1.61 

1.61 

1.92 

1.92 

1.76 

1.96 

1.62 

1.67 

1.31 

1. " 

2. ^3 
1.00 
1.00 
3.90 - 
3.90 
0.96 
0.35 
0.91 
1*06 
1.17 
0.19 
1.30 
2.05 

1.12 
1.03 

2.1^ 
1.16 
1.63 
1.22 
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1.61 
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0.96 . 

1.29 

0.03 

0.91 

1.00 

0.60 

1.00 

0.73 

1.11 

1.29 

0 c2 

1.10 

1.12 

0.09 

1.11 

1.03 

1.03 

1.27 

1.03 

1.11 

0.93 

0.93 

1.01 

1.01 

1.07 

0. 96 

1. :i 

1.12 
0.07 
0.93 
0.77 
0.61 
0.01 
1.00 
1.00 
1.10 
0.97 
1.10 
0.61 
1.23 
1.03 
1.10 
0.01 
1.13 
0.93 
O.0A 
1.51 
1.21 
1.00 
1*19 
1.11 
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1.36 

1.07 

0.53 

1.11 

1.10 

1.21 

0.70 

0.07 

1.39 

0*09 

0.15 

1.31 

3.23 

1.99 

3.19 

0.01 

0.01 

2.26 

1.26 

•>.30 

1.56 

1.56 

2.03 

2.05 

1.76 

1.95 

. 1.00 
2.17 
0.09 
1.21 
2.06 
0.05 
0.&5 
1.16 
1.16 
1.06 
0.31 
1.00 
0.90 
1.16 
0.51 
1.51 
".31 
1*2B 
0.97 
2.19 
1.76 
1-99 
1.23 
1.05 
1.90 



NTC 
BRR SI) 

1.39 

o.ftft 

0.17 

1.25 

1.27 

1.29 

0.S1 

1.07 

1.11 

0.57 

0.50 

1.27 

2.76 

2.26 

2.63 

0.01 

0.01 

1.00 

0.96 

1.05 

1.63 

1.63 

2.21 

2.21 

1.02 

1.05 

1.65 

2.10 

0.91 

W13 

2.17 

1.01 

1.01 

1.61 

1.61 

0.95 

0.33 

0.90 

1.01 

1.13 

0.19 

1.30 

2.79 

1.13 

1.03 

2.37 

1.16 

1.63 

1.22 

0.09 

1*59 
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SO RATIO 

0.97 

1.20 

1.11 

0.90 

1.16 

0.95 

0.65 

0.01 

0.95 

1.31 

0.77 

1.03 

1.16 

0.07 

1.21 

1.02 

1.02 

1.21 

1.31 

1.2ft 

0.91 

0.91 

0.92 

0.92 

0.96 

1.03 

1.11 

1.03 

0.97 

0.07 

0.03 

0.01 

0.01 

0.09 

0.09 

1.10 

0.97 

1.11 
0.07 
1.26 
1.02 

1.11 
0.03 
1.13 
0.93 
0.92 
1.51 
1.22 
1.00 
1.1ft 
1.19 
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SIZE 
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357 
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253 
255 
107 
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233 
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61 
61 
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357 
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357 
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103 
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1)3 
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Y-TRUE 

66,74 
25,26 
4,54 
15*60 
64.72 
42.63 
7.36 
11.23 
56.17 
13.06 
2.73 
26.03 
37.05 
7.36 
55.57 
92.75 
7,25 
32.20 
4.26 
6 3.52 
30.74 _ 
69.26 
49.04 
50.96 
20.16 
.32.46 • 
47.36 
37.64 
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3.61 
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3.59 
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1.91 

0,19 
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0.30 

4.98 

4.06 

5.53 

3. 12 

3.62 

3.62 



NI 


NI 


BRR SO 


SO RATIO 


3.63 


0.62 


3.26 


0.90 


1.10 


1.09 


1.93 


0.96 


3.37 


0.90 


2.76 


1.04 


1.1* 


1.34 


1.25 


1,06 


3.31 


0.97 


2.56 


0.82 


1.49 


0.83 


2.53 


1.11 


7.50 
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0.69 


1.69 
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1,62 

4,12 

2,33 

4,57 

2,62 

2,62 

3,32 

3,92 

3,47 

3,64 

3,90 

3,67 

2,19 

2,03 

3,90 

2,65 

2,65 

7,69 

7,69 

1.91 

0,19 

1.89 

1.77 

3.01 

1.29 

2.76 

5.21 

2 W 93 

0.30 

4.96 

4.08 

5.53 

3.12 

3.62 

3,62 



7. 



NIC 
BRR SO 

2.92 
3,39 
1,00 
2,31 
3.79 
2.67 
1-28 
1.29 
3.34 
2,66 
1.55 
2.76 
6.56 
.76 
.12 
1.72 
1.72 
3,95 
3.66 
4,14 
5.61 
3,61 
3,66 
3,68 
3.63 

H. 99 
3.30 
3,02 

I, 34 
2.08 
3,26 
3.61 
3.61 
9,26 
9,28 
1.91 
0,20 
1,78 
1.77 
2.62 
1,31 
2,32 

.06 
,45 
.30 
.49 
.9? 
.22 
.61 
3.61 
3.63 



6, 
3, 
0, 
4. 
3. 
3, 
2. 



NIC 
SO RATIO 

1.04 
0.6V 
1.1H 
0,68 
0.69 
1.17 
1.21 
1.12 
0.95 
0,76 
0,63 
1,05 
0,69 
0,63 
0,94. 
1,05 
1,05 
1,04 
0,63 
1*10 
0,77 
0,77 
1,06 
,06 
,90 
0,72 
.18 
,27 
,64 
.97 
.16 
,73 
,73 

• 62 
.62 
,00 

• 94 
.06 
,00 
,06 , 
,96 
.20 
.66 
,64 
,00 
.10 
.03 
.71 
.10 
.95 
,05 



1. 
0. 



1, 
1. 
1, 
0. 
1. 
0, 
0, 
0, 
0, 

1, 

0, 

1, 
1. 
1, 

0, 

1. 

0. 
0, 

1. 
1 . 
1 

1. 
1, 

0. 

1 . 



« 



Comparison of variance estimators for continuous items for the total population 





SAMPLE 




utri 


SIZE 


Y-TRU£. 


TU15 


11445 


38.75 


T016 


11445 


160.26 


TU89MA 


7579 


2102 # 30 


T009HU 


7579 


2173.09 


TU1H1FA 


15089 


7039.68 


T01H1FB 


15089 


8704. HI 



NI 


NI 


NI 


STDERR SO 


BRR SO 


SO RATIO 


D.13 


0.13 


1.03 


1 .13 


0.99 


i.ih 


27. 3** 


28.71 


0.95 


34.57 


3*4. 52 


1.00 


68.72 


69. 64 


0.98 


73.70 


75.88 


0.97 



'NIC 


NIC 


NIC 


STOERR SO 


BRR SO 


SO RATIO* 


0.13 


0.13 


1.00 


1.12 


1.00 


1.12 


26.32 


27.87 


0.94 


28.31 


20.08 


1.00 


67.80 


72.00 


0.94 


73.04 


68.48 


1.06 



<>f!0 



Comparison of variance estimators fofNcontinuous items for males 



1TEH 


SAHPLt 

size 


Y-TR *e 


TQ15 

TU16 

TQ69HA 

TQ69HB 

TQ141FA 

TQ141FB 


6062 
9970 
3970 
7357 
7357 


41 ,01 
163,64 
2162.70 
2904,56 
6623.30 
6214.16 



NI 

STOtRR SD 


NI 
BRR SD 


^ NI 
SD RATIO 


NIC 
STDERR SD 


NIC 
8RR SD 


NIC 
SO RATIO 


0.16 
1.74 
39,29 
53.62 
61,22 
94.61 


0.19 
1,65 
42.50 
53,63 
92.31 
104,20 


0,93 
1.05 
0.92 
0.99 
0.67 
0.90 


0,16 
1.72 
37.05 
40.51 
76.99 
92.19 


0.19 
1.72 
34.33 
29,30 
92.90 
106,16 


0 r 92 
U00 
1.07 
1,36 
0.32 
0.66 
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Comparison of variance estimators for continuous items for individuals of high ability 





SAMPLE 




ITEM 


SIZE 


Y-TRUE 


TQ15 


2107 


36.26 


T016 


2107 


1H0. 26 


TU89HA 


2155 


2597. 


TU69MB 


2155 


2696,50 


TQ1H1FA 


2839 


5327.10 


TQ1HIFB 


2039 


73H9.55 



* 



HI MI Nl 

ST0ERR SO BRfl SO SO RATIO 

0.33 0.31 1*06 

2.Q6 1.65 1*12 

H8.59 H9.95 0.97 

55.71 55.99 0.99 

115. 5H 110.92 1.0H 

132. «*5 11H.10 1.16 



NIC 


NIC 


NIC 


STOERR SO 


BRR SO 


SO RATIO 


0,33 


0.31 


1.C6 


2.08 


1.93 


1.07 


«*H»09 


31.33 


1,40 


52.27 


39.63 | 


1.31 


116.15 


107.98 


1.07 


126. 9H 


112.81 / 


i.M 



2H7 



prison of variant es tIm ator, for continuous ,«« for Individuals of 1« socio-scoooMc status 



ITEH 

TQ15 

TU16 

TQ69HA 

T089HB 

TUl*lM 



SAMPLE 
SIZE 

3151 
3151 
1*20 
1*20 
1220 
1220 



Y-TRUE 

39. 
156. 5* 
1636.01 
1732. 6* 
7662.66 
8966.36 



NI 

ST0EKR SO 

0.21 

61.21 
77.27 
116.17 
12*. *9 



NI 

BRR SO 


NI 

SO HAT 10 


NIC 
ST0ERR SO 


NiC 
BRR SO 


NIC 
SO RATIO 




0.12 
2.16 
53.59 
65.25 
125.03 
139.5* 


1.75 
0.96 
1.19 
1.16 
0.9* 
0.69 


0.21 
2.15 
61.3* 
66.02 
11**26 
12*. 95 


0.12 
2.26 
35.71 
51.75 
102.60 
122.96 


1.66 

0. 9* 
1.71 
1.31 

1. " 
1.01 
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Comparison of variance estimators for continuous items for blacks 





SAHPU 




NI 


NI 


NI 


NIC 


NIC 


NIC 


IUM 


SIZE 


Y-TRUE 


STOERR SU 


BRR SO 


SO RATIO 


STOERR SO 


BRR SO 


SO RATIO 


TQ13 


30HO 


39.06 


0.27 


0.23 


1.16 


0.27 


0.26 


1.0*1 


TU16 


30*10 


162.99 


1.89 


1.60 


1.05 


1.92 


1.9*1 


0.98 


TU89HA 


1922 


2127.80 


*I7.39 


3S.32 


1.20 


H9.32 


H3.08 


1.1*1 


TU89HB 


1922 


2196.52 


53.19 


58.81 


0.90 


50.89 


53. 6<l 


0,9*1 


T01«UFA 


39*18 


7352.66 


130.70 


83.52 


1.56 


127.61 


91.01 


l.*«0 


T01«*lF8 


39H8 


9119.38 


136.50 


112. 5ft 


1.21 


13M.12 


105.22 


1.27 



Comparison of variance estimators for continuous items for the 





SAflPLF. 




lien 


SIZE 


t-TRUt 




1H56 


37. *e 


TUlb 


1*56 


H5.73 


TU69HA 


928 


1702.03 


TU69HB 


925 


177*. 33 


TQH1FA 


1963 


39H5.ee 


TU101FB 


1963 


6969.37 



NI NI NI 

STDCRR SO nRR SO SD RATIO 

0.33 0.26 i«t6 

3.07 3.01 0.00 

90.*»6 61.27 1*11 

113. 2* 110.13 1.02 

171. *2 17H.50 0.96 

19c. 03 20H,?0 0.9b 



NIC 


NIC 


NIC 


STOCilR SO 


BRR SU 


SO RATIO 


0.33 


0.27 


5 i.ie 


3.11 


3.60 


0.61 


76.07 


69.57 


1.12 


66.14 


6H.69 


1.33 


176.H3 


191.7** 


0.92 


203.59 


£22.63 


0.91 



SO 
oo 
t 
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Comparison of variance estimators for continuous items for blacks of average ability 





SAMPLE 




NI 




HEM 


SIZE 


Y-TRUi 


STOEHR SO 


0RR SO 


TU15 


256 


37.11 


0.69 


0.73 


1016 


256 


162. e-i 


11.22 


12.66 


TU89HA 


233 


1690. IB 


1*42.68 


116.93 


TU89HB 


233 


1974.43 


150 . 04 


143.32 


TU141FA 


357 


56*48,06 


376.67 


406.61 


tumiFB 


357 


6901.88 


414 .62 


483.29 



NI 


NIC 


NIC 


NIC 


SO RATIO 


STOERR SO 


BRR SO 


SO RATIO 


0.93 


0.70 


0.75 


0.94 


0.87 


11.37 


13*21 


0.86 


1.22 


14^.32 


133.79 


I. 11 


1 ,04 


158.64 


153.65 


1.03 


0.92 


374.58 


420.10 


C.89 


0.65 


419. 66 


485.84 


C.86 



VO 
I 



27 i 



9 

ERIC 



